Subject: [dmarc-ietf] Fwd: New Version Notification for draft-kucherawy-dkim-transform-01.txt

From: "Murray S. Kucherawy" <[email-masked]>
Date: Sun, 5 Jul 2020 14:56:26 -0700

I decided to breathe life into this idea since it's relevant and got some
discussion recently.  Comments welcome.

I'm talking to the Mailman people about the idea now; this is based on some
things they mentioned.  I haven't managed to get the attention of Sympa or
L-Soft yet.

-MSK

---------- Forwarded message ---------
From: <[email-masked]>
Date: Sun, Jul 5, 2020 at 2:46 PM
Subject: New Version Notification for draft-kucherawy-dkim-transform-01.txt
To: Murray S. Kucherawy <[email-masked]>

A new version of I-D, draft-kucherawy-dkim-transform-01.txt
has been successfully submitted by Murray Kucherawy and posted to the
IETF repository.

Name:           draft-kucherawy-dkim-transform
Revision:       01
Title:          Recognized Transformations of Messages Bearing DomainKeys
Identified Mail (DKIM) Signatures
Document date:  2020-07-05
Group:          Individual Submission
Pages:          14
URL:
https://www.ietf.org/internet-drafts/draft-kucherawy-dkim-transform-01.txt
Status:
https://datatracker.ietf.org/doc/draft-kucherawy-dkim-transform/
Htmlized:
https://tools.ietf.org/html/draft-kucherawy-dkim-transform-01
Htmlized:
https://datatracker.ietf.org/doc/html/draft-kucherawy-dkim-transform
Diff:
https://www.ietf.org/rfcdiff?url2=draft-kucherawy-dkim-transform-01

Abstract:
   DomainKeys Identified Mail (DKIM) introduced a mechanism whereby a
   mail operator can affix a signature to a message that validates at
   the level of the signer's domain name.  It specified two possible
   ways of converting the message body to a canonical form, one
   intolerant of changes and the other tolerant of simple changes to
   whitespace within the message body.

   The provided canonicalization schemes do not tolerate changes in a
   message such as conversion between transfer encodings or addition of
   new message content.  It is useful to have these capabilities to
   allow for transport through gateways, and also for transport through
   handlers (such as mailing list services) that might add content that
   would invalidate a signature generated using the existing
   canonicalization schemes.

   This document presents a mechanism for declaring that a message
   underwent one of a handful of well-defined transformations prior to
   being re-signed by a mediator, so that a verifier might rewind such a
   modification and thereby confirm that the original signature still
   verifies against the original content.

Please note that it may take a couple of minutes from the time of submission
until the htmlized version and diff are available at tools.ietf.org.

The IETF Secretariat

---

From: Jeremy Harris <[email-masked]>
Date: Sun, 5 Jul 2020 23:31:24 +0100

> https://tools.ietf.org/html/draft-kucherawy-dkim-transform-01

Substansive:

section 6, on footers:
- Is not a common boundary marker a signature-marker, namely
  two dashes, space, end-of-line?  The exim-users list, for
  example, uses that.

Nit:

section 3:   s/such transmission was/such transformation was/

-- 
Cheers,
  Jeremy

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Sun, 5 Jul 2020 15:39:33 -0700

>   two dashes, space, end-of-line?  The exim-users list, for

Yes, that's common, but I also looked at a list run by Mailman and found
that it uses a line of underscores.  I thought having one transformation
that handles either case might be a good idea, but I'm happy to split them
or whatnot if they can both be precisely described and that's what people
want to do.

Any opinion on the whole thing generally?

> section 3:   s/such transmission was/such transformation was/

Got it!

-MSK

---

From: Jeremy Harris <[email-masked]>
Date: Sun, 5 Jul 2020 23:48:39 +0100

> Any opinion on the whole thing generally?

Certainly worthy of discussion.  I wonder if it needs tying
to ARC rather than, or in addition to DKIM?
-- 
Cheers,
  Jeremy

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Sun, 5 Jul 2020 15:53:14 -0700

> to ARC rather than, or in addition to DKIM?

I think it solves the same problem ARC does.  You want to know what
authentication was possible at the time the MLM received the message; ARC
transmits that information downstream using cryptographic signatures, while
this lets the verifier see for itself.

I originally floated this idea years before ARC was proposed, but there was
no apparent community interest in pursuing it so I let it expire.

-MSK

---

From: "John Levine" <[email-masked]>
Date: 5 Jul 2020 22:42:51 -0400

In article <[email-masked]> you write:
>-=-=-=-=-=-

I wrote the Sympa ARC code which is entwined with the DKIM code so
that would probably be me. Honestly, this looks like a lot more work
than ARC to get a result likely to be worse in practice than ARC.

It would not be hard for a bad guy to use the footer or add-part
transformation to lay a big spammy blob on top of some innocuous
original message. Rather than play cat and mouse and try to figure one
when a change is too big, recipients would use this the same way they
use ARC, and only check it on mail from senders who are generally well
behaved.

I believe that the main thing that recipients will do with either of
these is to ask was the original message actually from the putative
sender, and ARC is a lot easier to implement.

R's,
John

---

From: Hector Santos <[email-masked]>
Date: Sun, 05 Jul 2020 23:35:31 -0400

1) Curious, are these Mail List Server (MLS) developers active 
participants of the WG list?  Lurkers?  Was there a consideration to 
include a MLS developer participant that is active in the WG? I'm sure 
you are aware SSI (Santronics Software, Inc) has a MLS albeit 
commercial, not free, not open source.

2) Was Mailman asked (the other two MLS were not) if a simple DNS 
lookup can be considered? IMO, as a MLS developer, it would be less 
coding than making a more complex change with this proposal or ARC. 
Given the proposal would be asking MLS developers to finally make 
changes to their code, it opens an opportunity to explore other 
available well-defined, empirically proven to work "running code" 
options.

3) And this hurts to ask, but I believe it is a natural "IETF Discuss" 
question to ask, is there an AD vs Editor conflict here? It is more of 
a general question and not specifically address to you as WG AD and 
the proposal editor. I'm sure you can be fair, but nonetheless, I have 
an active implementer "fairness" concern with each of these itemize 
questions.

Thanks

-- 
Hector Santos,
https://secure.santronics.com
https://twitter.com/hectorsantos

On 7/5/2020 5:56 PM, Murray S. Kucherawy wrote:
> I decided to breathe life into this idea since it's relevant and got
> some discussion recently.  Comments welcome.
>
> I'm talking to the Mailman people about the idea now; this is based on
> some things they mentioned.  I haven't managed to get the attention of
> Sympa or L-Soft yet.
>
> -MSK
>
> From: <[email-masked] <mailto:[email-masked]>>
> To: Murray S. Kucherawy <[email-masked] <mailto:[email-masked]>>
>
>
>
> Title:          Recognized Transformations of Messages Bearing
> DomainKeys Identified Mail (DKIM) Signatures
> Status: https://datatracker.ietf.org/doc/draft-kucherawy-dkim-transform/
> Htmlized: https://tools.ietf.org/html/draft-kucherawy-dkim-transform-01
> Htmlized:
> Diff: https://www.ietf.org/rfcdiff?url2=draft-kucherawy-dkim-transform-01
>
> Abstract:
> until the htmlized version and diff are available at tools.ietf.org
> <http://tools.ietf.org>.
>
> The IETF Secretariat
>
>
>
>
> _______________________________________________
> dmarc mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/dmarc
>

-- 
Hector Santos,
https://secure.santronics.com
https://twitter.com/hectorsantos

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Sun, 5 Jul 2020 21:43:53 -0700

> commercial, not free, not open source.

The ones I've approached are not here, so that's why I reached out to
them.  You, on the other hand, are already participating, and are free to
give your perspective.

> available well-defined, empirically proven to work "running code"

No, I have not put that question to them.  If they join the list, you could
ask them here; if they don't, you can reach out to them to make that
inquiry.

> an active implementer "fairness" concern with each of these itemize

Fair question.

Area directors are not barred from participating in working groups.  If the
WG were to choose to adopt this draft, for example, the chairs would have
the choice of picking a different editor, as they do with any other
document.  It would be the chairs that make consensus calls as needed as
the document evolves.  They run the Working Group Last Call.  Once it gets
sent up for IETF-wide Last Call, some other Area Director would step in to
run it through that part of the process (probably Barry, the other ART AD,
but not necessarily).  And when it gets to IESG balloting, I would recuse
myself.  If at any point it looks like a conflict of interest is possible,
we tag other Area Director(s) to review it for undue influence.

-MSK

---

From: Jim Fenton <[email-masked]>
Date: Sun, 5 Jul 2020 21:50:45 -0700

> use ARC, and only check it on mail from senders who are generally well

That was basically the argument against the l= parameter in DKIM
signatures. We did end up keeping l= because it only has effect if the
signer uses it and the verifier accepts its use, although it was widely
expected that it would not be used much. I suspect that's what happened.

-Jim

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Sun, 5 Jul 2020 22:00:59 -0700

> than ARC to get a result likely to be worse in practice than ARC.

Interesting; I had the opposite experience, and I had a lot of our DKIM
code to recycle when I built our ARC implementation.  This idea doesn't
seem nearly as complicated to implement.  Moreover, ARC is a much more
heavyweight addition to the stack than a new DKIM tag would be.

> use ARC, and only check it on mail from senders who are generally well

That isn't a new attack though, given spammers sign their email already.
This way you have (in theory) two good signatures: one from the author for
the "safe" form of the message, and one for the spam that got bolted on,
and you could in theory strip the spam before you deliver because you know
exactly what it is.

> sender, and ARC is a lot easier to implement.

As I said upthread, I agree that it fulfills the same need that ARC does.
There was a previous thread about ways to avoid MLMs needing to munge
headers in a DMARC world, and that reminded me of this, so I thought it
might be worth dusting it off.

-MSK

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Sun, 5 Jul 2020 22:04:32 -0700

> expected that it would not be used much. I suspect that's what happened.

That sounds right to me.  OpenDKIM, for example, makes it optional to
consider signatures using "l=" invalid, or that have more than N bytes
added using "l=", etc.

-MSK

---

From: Hector Santos <[email-masked]>
Date: Mon, 06 Jul 2020 01:22:06 -0400

Yes, I was aware an AD can participate in a WG, but was not sure if 
the AD can do so in a WG where there might be proposed working items 
authored by the AD. A recusal consideration for a conflict, if any, 
would be the proper recourse.

Thanks for your fair responses. I appreciate it.

On 7/6/2020 12:43 AM, Murray S. Kucherawy wrote:
> On Sun, Jul 5, 2020 at 8:35 PM Hector Santos <[email-masked]
> <mailto:[email-masked]>> wrote:
>
> them.  You, on the other hand, are already participating, and are free
> to give your perspective.
>
> No, I have not put that question to them.  If they join the list, you
> could ask them here; if they don't, you can reach out to them to make
> that inquiry.
>
> Area directors are not barred from participating in working groups.
> If the WG were to choose to adopt this draft, for example, the chairs
> would have the choice of picking a different editor, as they do with
> any other document.  It would be the chairs that make consensus calls
> as needed as the document evolves.  They run the Working Group Last
> Call.  Once it gets sent up for IETF-wide Last Call, some other Area
> Director would step in to run it through that part of the process
> (probably Barry, the other ART AD, but not necessarily).  And when it
> gets to IESG balloting, I would recuse myself.  If at any point it
> looks like a conflict of interest is possible, we tag other Area
> Director(s) to review it for undue influence.
>
> -MSK
>
>

---

From: Alessandro Vesely <[email-masked]>
Date: Mon, 6 Jul 2020 10:49:06 +0200

> expected that it would not be used much. I suspect that's what happened.

The rationale for deprecating l= was that HTML and CSS allow appended content 
to be displayed before or even above (overlapping) the original content.

I don't recall an actual proof of concept, though.  Perhaps, we could have 
thought better:

1.  In most cases, text/html is relegated in its own MIME entity.  Multipart 
messages don't support simple append, except for the "epilogue" area.

2.  In any case, the </html> etag is covered by the signature.  Added text 
would trigger quirks mode, and suppress CSS.

Rather than deprecate l=, perhaps we could just have noticed that it only makes 
sense in text/plain messages, where it allows the addition of a footer.  For 
example, this message —now that IETF seems to have given up base64 encoding— 
should feature an unbroken author signature.

For multipart messages, transformations may need to replace boundaries.  In 
that case, the "restricted patch" to undo the transformation may require more 
data than it is convenient to store in a DKIM tag.

Best
Ale
--

---

From: Alessandro Vesely <[email-masked]>
Date: Mon, 6 Jul 2020 11:03:33 +0200

On Mon 06/Jul/2020 10:49:06 +0200 Alessandro Vesely wrote:
> example, this message [...]now that IETF seems to have given up base64 encoding[...] 

It doesn't.  It was base64 encoded.  Perhaps because of non-ASCII characters 
(which I elided in the quotation above)?

Best
Ale
--

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Mon, 6 Jul 2020 02:24:19 -0700

> data than it is convenient to store in a DKIM tag.

Replace why?  It might need to add its own, but it's easy to undo that.

In order to be useful, this only needs to do what MLMs commonly do
already.  We don't need to cover the universe of possible futures right
away.

-MSK

---

From: "Doug Foster" <[email-masked]>
Date: Mon, 6 Jul 2020 08:48:48 -0400

I like the idea of making signatures recoverable wherever possible.

For mailing lists, I wonder if this approach is sufficient to solve the whole problem.   If the MLM  transformation is for security rather than informational purposes, I expect that the  transformations will be (even should be) non-reversible. 

For some spam filters, it might be important.   Lots of spam filters add DKIM-invalidating content upon entry to the protected domain.   Many of those changes should be reversible.  URL rewrite is the most difficult to reverse using this approach.   

However, when the incoming and outgoing email gateways are from the same vendor, as I suspect is often the case, the reversal should already be feasible using proprietary methods.   Is any known spam filter vendor doing this type of reversal?

DF

From: dmarc [mailto:[email-masked]] On Behalf Of Murray S. Kucherawy
Sent: Sunday, July 05, 2020 5:56 PM
To: IETF DMARC WG
Subject: [dmarc-ietf] Fwd: New Version Notification for draft-kucherawy-dkim-transform-01.txt

I decided to breathe life into this idea since it's relevant and got some discussion recently.  Comments welcome.

I'm talking to the Mailman people about the idea now; this is based on some things they mentioned.  I haven't managed to get the attention of Sympa or L-Soft yet.

-MSK

---------- Forwarded message ---------
From: <[email-masked]>
Date: Sun, Jul 5, 2020 at 2:46 PM
Subject: New Version Notification for draft-kucherawy-dkim-transform-01.txt
To: Murray S. Kucherawy <[email-masked]>

Name:           draft-kucherawy-dkim-transform
Revision:       01
Title:          Recognized Transformations of Messages Bearing DomainKeys Identified Mail (DKIM) Signatures
Document date:  2020-07-05
Group:          Individual Submission
Pages:          14
URL:            https://www.ietf.org/internet-drafts/draft-kucherawy-dkim-transform-01.txt
Status:         https://datatracker.ietf.org/doc/draft-kucherawy-dkim-transform/
Htmlized:       https://tools.ietf.org/html/draft-kucherawy-dkim-transform-01
Htmlized:       https://datatracker.ietf.org/doc/html/draft-kucherawy-dkim-transform
Diff:           https://www.ietf.org/rfcdiff?url2=draft-kucherawy-dkim-transform-01

Abstract:

The IETF Secretariat

---

From: Benny Pedersen <[email-masked]>
Date: Mon, 06 Jul 2020 15:09:13 +0200

Doug Foster skrev den 2020-07-06 14:48:
> For mailing lists, I wonder if this approach is sufficient to solve
> the whole problem.   If the MLM  transformation is for security rather
> than informational purposes, I expect that the  transformations will
> be (even should be) non-reversible.
> 
> For some spam filters, it might be important.   Lots of spam filters
> add DKIM-invalidating content upon entry to the protected domain.
> Many of those changes should be reversible.  URL rewrite is the most
> difficult to reverse using this approach.
> 
> However, when the incoming and outgoing email gateways are from the
> same vendor, as I suspect is often the case, the reversal should
> already be feasible using proprietary methods.   Is any known spam
> filter vendor doing this type of reversal?

i know mailman can do the right things, ARC sealing, and then nothing 
more, then its op to DMARC to test results from ARC, job done

but this does not work if DKIM is breaked before ARC sealing is done, 
and DMARC testing is not yet maked into stable releases yet, and in 
gentoo none of it exists

i have dropped OpenDKIM, OpenARC, OpenDMARC, and now just live with 
fuglu

>    verifies against the original content.

in the end it might work better if maillists in generic does not try to 
fix anything thay self creates as a problem to solve, dovecot and 
postfix maillists is known to not break DKIM, and currrently only 
dovecot is doing ARC sealing, not the worst case, but nice to know that 
maillist can stop breaking DKIM

hope it will be standard in 2021 finaly

---

From: "John R Levine" <[email-masked]>
Date: 6 Jul 2020 10:31:13 -0400

> heavyweight addition to the stack than a new DKIM tag would be.

I was thinking of the work involved in putting it into the mailing list 
software.  ARC was straightforward, one bit where the messages come in to 
recard the A-R header, another bit cloned from the DKIM code to apply the 
seal.

For this I'd need to go through vast amounts of code looking for every 
place that changes the message and figuring out what change might map onto 
what DKIM key, and we still have a lot of changes that don't map onto 
anything, so now some changes are safer than others.

>> transformation to lay a big spammy blob ...

> and you could in theory strip the spam before you deliver because you know

But then what?  Surely we're not going to revisit the WKBI of showing 
different parts of the message in different colors depending on whether 
they're signed.  If it's got bolted on spam, you treat the message as spam 
so I don't see that this has gained you anything.

R's,
John

---

From: "Doug Foster" <[email-masked]>
Date: Mon, 6 Jul 2020 11:43:27 -0400

If MLM changes are not reversible, we still have the problem of convincing the recipient gateway to trust the modified message.

At first glance, this is an internal issue between the user who created the subscription and his email security administrator, and that communication is not obviously an IETF problem.    But in a large consumer-focused environment like Gmail, or even within a Fortune 500 enterprise, some level of automation may be necessary.   Additionally, the email filter MTA and the mail store are often separate products from separate companies, so we have an multi-vendor interoperability issue.  Collectively this seems to make an IETF specification appropriate.

At present, email gateways have very limited ability to distinguish between solicited and unsolicited mail.   Some implementations check for a match on the user’s contact list.   Other products check for outbound messages to the sender.    Giving the email gateway better information about subscribed mail flows should improve filtering accuracy.   

This is the scenario in my head, a variant of something posted previously:

·         User A subscribes to mailing list M.

·         Mailing list M sends a message to User A with a request to ensure that the inbound gateway will accepts its message.   The message contains attachments needed to define and justify the request.    I envision an XML attachment for use with automation, and a text file for non-automated environments.   An XLST file could be used to convert the XML to TXT, to validate that the XML attachment and the text attachment are really stating the same thing.   

·         The configuration files provide this type of information:

o   The header fields which will uniquely identify message from this subscription, and the tests which can be performed to validate those headers.

o   The administrative contacts for the list.

o   Whether the list makes DKIM-invalidating changes, and why.

o   For Murray’s approach:   Whether the changes will be fully or partially reversible and whether it supports the tf extension.

o   For John’s approach:  Whether or not the list implements ARC.

·         The email administration then reviews the request and replies to user and MLM in one of three ways:

o   Fully Approved - any required gateway trust changes have been implemented.  For automated systems, the XML file is used to apply these changes.

o   Strongly rejected  - subscription is contrary to policy and should be cancelled; user should assume that any incoming messages will be blocked.

o   Neutral – Subscription is not rejected but no trust changes will be made.   Any particular message may or may not be blocked.

Would this be more likely to succeed than the alternatives?

DF

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Mon, 6 Jul 2020 09:44:01 -0700

> so I don't see that this has gained you anything.

No, I'm not saying render them differently.  I'm saying that if the second
signature passes, then the second one signed the bolted-on spam but also
told you how to strip it away to get the original.  So, do that; if the
author signature now passes, you have the original "clean" message to show
instead of the hijacked message.  If not, you have a spammy message to deal
with, as before.

If the second signature doesn't pass in the first place, then it's ignored,
and you still have spam to deal with.

Of course, the original message might be spam too, but that's not new
either.

This isn't meant to solve spam.  It's meant to deal with the legitimate
MLM+DKIM+DMARC case.

-MSK

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Mon, 6 Jul 2020 09:49:11 -0700

On Mon, Jul 6, 2020 at 8:43 AM Doug Foster <[email-masked]>
wrote:

> ·         Mailing list M sends a message to User A with a request to
> ensure that the inbound gateway will accepts its message.   The message
> contains attachments needed to define and justify the request.    I
> envision an XML attachment for use with automation, and a text file for
> non-automated environments.   An XLST file could be used to convert the XML
> to TXT, to validate that the XML attachment and the text attachment are
> really stating the same thing.
>

The first thing that comes to mind for things like this is whether and how
spammers will try to use this to clear a path to the inbox.  How do you
know this XML blob is coming from a bona fide list?

Anecdotally, spammers were the first to adopt DKIM.  This wouldn't be any
different.

·         The email administration then reviews the request and replies to
> user and MLM in one of three ways:
>

This needs to be automatable, or it won't scale to very large operations.
I suspect there's virtually zero chance a GMail sized operation will staff
up to review requests given the size of their customer base.

And once it's automated, there needs to be a strategy for dealing with
stale data (e.g., old lists nobody is using anymore), etc.

-MSK

---

From: "John R Levine" <[email-masked]>
Date: 6 Jul 2020 13:03:07 -0400

> instead of the hijacked message.  If not, you have a spammy message to deal

I don't understand this scenario at all.  Why would I want to show my user 
a message forwarded by a spammer?  If the original sender wanted me to see 
it, she could have sent it to me directly, or through a legit mailing 
list.

R's,
John

---

From: Alessandro Vesely <[email-masked]>
Date: Mon, 6 Jul 2020 19:08:22 +0200

On Mon 06/Jul/2020 11:24:19 +0200 Murray S. Kucherawy wrote:
>> For multipart messages, transformations may need to replace boundaries. 
>> In that case, the "restricted patch" to undo the transformation may
>> require more data than it is convenient to store in a DKIM tag. >>
> 

Oops, I thought MLM generated boundaries anew.  It seems that's not the case.

Still, the original Content-Type is difficult to reproduce exactly.  You need 
to know whether the boundary= parameter is written as a token or a 
quoted-string, and, if not c=relaxed, spaces and newlines.  If longer than a 
few bits, the info to undo the transformation (a reverse patch?) could be 
stored in its own field, or even in an attachment.

Another difficulty, IIRC, is reordering of To:.  This can simply be avoided.

> already.  We don't need to cover the universe of possible futures right

Agreed.  MLMs code has no lack of conditionals.  Probably this technique can 
address a class of mailing lists somewhat wider than the "no-changes" class, 
without trying to cover even one half of /present/ MLM possibilities.  Shall 
say just enough to cover IETF mailing lists?

Best
Ale
--

---

From: Dave Crocker <[email-masked]>
Date: Mon, 6 Jul 2020 10:12:05 -0700

On 7/6/2020 10:03 AM, John R Levine wrote:
> I don't understand this scenario at all.  Why would I want to show my 
> user a message forwarded by a spammer?  If the original sender wanted 
> me to see it, she could have sent it to me directly, or through a 
> legit mailing list. 

Perhaps, like some others, I'm not understanding this correctly, but I 
think the proposal has nothing at all to do with what the recipient 
sees.  Rather, I've understood this as an attempt to reverse additions 
made by a Mediator, with the goal of validating the origination DKIM 
signature.  Presumably that is so as to use the origination domain's 
reputation and even permit DMARC to validate.

d/

-- 
Dave Crocker
Brandenburg InternetWorking
bbiw.net

---

From: Alessandro Vesely <[email-masked]>
Date: Mon, 6 Jul 2020 19:21:46 +0200

> up to review requests given the size of their customer base.

It can be automated by having logged-in users interact with a suitable 
server-side program.  That approach can work for (small) mailbox providers 
which trust their users.  That set seems to be close to the complement of the 
ones who can use ARC, doesn't it?

> stale data (e.g., old lists nobody is using anymore), etc.

Yes, it would be an improvement of the current subscription reminders —a time 
distributed database.

Best
Ale
--

---

From: [email-masked]
Date: Mon, 06 Jul 2020 10:17:42 -0700 (PDT)

> Oops, I thought MLM generated boundaries anew.  It seems that's not the case.

Depends on the MLM and what it is doing. For example, boundary marker
regeneration can happen if you're adding a header or a footer to a multipart
message.

> say just enough to cover IETF mailing lists?

The problem is that list software wasn't designed with the idea of keeping
changes to an invertible/removable minimum in mind. So you're going to see
many/most of the transformations this document deals with done in ways other
than what the document describes. So in order to make use of this document
you're basically talking about a fairly major rewrite.

I think that's unlikely to happen.

				Ned

---

From: "John R Levine" <[email-masked]>
Date: 6 Jul 2020 13:41:30 -0400

On Mon, 6 Jul 2020, Dave Crocker wrote:
>> it, she could have sent it to me directly, or through a legit mailing list. 
>
> Perhaps, like some others, I'm not understanding this correctly, but I think 
> the proposal has nothing at all to do with what the recipient sees.  Rather, 
> I've understood this as an attempt to reverse additions made by a Mediator, 
> with the goal of validating the origination DKIM signature.  Presumably that 
> is so as to use the origination domain's reputation and even permit DMARC to 
> validate.

But why would I want to do that?  ARC lets a credible mediator say this 
message was OK before I munged it.  This proposal lets a sleazy mediator 
say the same thing, with advice on how to verify mechanically.

A sleazy mediator takes a message from Paypal and wraps a big blob of HTML 
spam around it that will display on top of the original message.  I get 
the spammy message, look at the signatures and find yup, there's a real 
Paypal message inside the spam.  What should I do with it?  It's unlikely 
the Paypal message was intended for me.

Regards,
John Levine, [email-masked], Taughannock Networks, Trumansburg NY
Please consider the environment before reading this e-mail. https://jl.ly

---

From: Dave Crocker <[email-masked]>
Date: Mon, 6 Jul 2020 10:48:59 -0700

On 7/6/2020 10:41 AM, John R Levine wrote:
>> Perhaps, like some others, I'm not understanding this correctly, but 
>> I think the proposal has nothing at all to do with what the recipient 
>> sees.  Rather, I've understood this as an attempt to reverse 
>> additions made by a Mediator, with the goal of validating the 
>> origination DKIM signature.  Presumably that is so as to use the 
>> origination domain's reputation and even permit DMARC to validate.
>
> But why would I want to do that? 

I wasn't advocating or criticizing.  Just trying to synchronize that 
nature and purpose of the task.

> ARC lets a credible mediator say this message was OK before I munged it. 

That's a very different trust model from allowing a means to directly 
vet the original signature.

> This proposal lets a sleazy mediator say the same thing, with advice 
> on how to verify mechanically.

Actually, it doesn't.

The sleazy mediator cannot somehow forge an originator's signature so it 
validates.

> A sleazy mediator takes a message from Paypal and wraps a big blob of 
> HTML spam around it that will display on top of the original message. 
> I get the spammy message, look at the signatures and find yup, there's 
> a real Paypal message inside the spam.  What should I do with it?  
> It's unlikely the Paypal message was intended for me.

A worthy scenario to worry about, but completely different from the 
nature of what ARC does and its likely benefits and weaknesses.

d/

---

From: Dave Crocker <[email-masked]>
Date: Mon, 6 Jul 2020 10:51:59 -0700

It occurs to me that discussion about how this latest proposal might or 
might not have similarities to ARC should encourage everyone making a 
proposal to add commentary that gives a full sense of end-to-end use.

That is, distinguish the details of how the specific mechanism works, 
from how it integrates into end-to-end service.  How is it expected to 
get used and why is that believed to be practical (as well as useful?)

d/

---

From: "John Levine" <[email-masked]>
Date: 6 Jul 2020 14:18:47 -0400

>proposal to add commentary that gives a full sense of end-to-end use.

Agreed. Hence my point that this is much harder to integrate into
mailing lists, and the useful signal you get, that you started with a
valid message from some X, is the same.

R's,
John

---

From: Alessandro Vesely <[email-masked]>
Date: Mon, 6 Jul 2020 20:28:38 +0200

> regeneration can happen if you're adding a header or a footer to a multipart

So that kind is going to go among forbidden transformations.

> you're basically talking about a fairly major rewrite.

Yet, setting no-changes does guarantee no breakage occurs.  Hm... does it?  I 
didn't carry out thorough tests.

Looking at my own messages to this list, which are signed with l=, I see my 
DKIM signatures are broken intermittently.  It would be enough to turn off 
base64 encoding.  It is no major rewrite...

> I think that's unlikely to happen.

Which is probably why this draft was let expire.

Best
Ale
--

---

From: Jim Fenton <[email-masked]>
Date: Mon, 6 Jul 2020 11:33:14 -0700

On 7/6/20 10:41 AM, John R Levine wrote:
>>> I don't understand this scenario at all.  Why would I want to show
>>> my user a message forwarded by a spammer?  If the original sender
>>> wanted me to see it, she could have sent it to me directly, or
>>> through a legit mailing list. 
>>
> But why would I want to do that?  ARC lets a credible mediator say
> this message was OK before I munged it.  This proposal lets a sleazy
> mediator say the same thing, with advice on how to verify mechanically.

Your use of  "credible mediator" and "sleazy mediator" emphasizes that
we're depending on the mediator behaving responsibly. Given that's the
case, why not just expect a responsible mediator to verify the DKIM
signature (or maybe SPF) on the incoming message, check its alignment
with the From: domain, then make whatever modifications it wants to
make, then re-sign the message with the mediator's DKIM signature
containing a tag that says it did all of the above?

Yes, this is a "get out of DMARC free" card for mediators to use. But
we're already dependent on being able to distinguish between credible
mediators and sleazy mediators, and this tag simply says, "if you trust
that I'm a credible mediator and this message has a valid signature from
me, you should accept the message even if my signature doesn't align
with the From: domain."

This gets us out of the business of trying to define what acceptable and
unacceptable transformations are. If the transformation was done by a
credible mediator, it's acceptable.

Many (most?) mediators do not currently require authentication
(+alignment) on incoming messages. They could continue to forward the
unauthenticated messages, but without the new tag.

-Jim

P.S. I'm still not sold on the value of From: domain alignment, but left
that in here to avoid conflating too many different ideas.

---

From: "John Levine" <[email-masked]>
Date: 6 Jul 2020 22:41:18 -0000

In article <[email-masked]>,
>containing a tag that says it did all of the above?

According to people I've talked to about ARC, because mailing lists
don't do that. One of the things that makes it plausible that lists
might implement ARC is that it doesn't ask for any changes in the
internal operation of the list, just slap an ARC signature on the end.

It's also useful for other kinds of forwarding that don't change
anything but since they're forwards, SPF fails.

This proposal makes lists sort through all of the changes they make
and try to figure out which ones match a tag and which ones don't.
That is surprisingly hard, e.g., I found that when you have
multipart/alternative and add a message header, it edits the header
text into both of the alternative versions.  Good luck unscrambling that.

-- 
Regards,
John Levine, [email-masked], Primary Perpetrator of "The Internet for Dummies",
Please consider the environment before reading this e-mail. https://jl.ly

---

From: Jim Fenton <[email-masked]>
Date: Mon, 6 Jul 2020 16:27:35 -0700

On 7/6/20 3:41 PM, John Levine wrote:
>> Your use ofÂ  "credible mediator" and "sleazy mediator" emphasizes that
"Just slap an ARC signature on the end" greatly understates the
complexity of ARC.
> anything but since they're forwards, SPF fails.
If a mediator can add an ARC signature, they can add a DKIM signature.
All this would add to DKIM signing is an additional tag indicating
whether the message received by the mediator had an aligned signature or
SPF.
> text into both of the alternative versions.  Good luck unscrambling that.

Perhaps I didn't explain this clearly enough. Mediators don't need to
sort through changes at all. All they do is check to see if the incoming
message had an aligned signature or SPF, and include a tag in the DKIM
signature that they apply indicating that.

Receiving domains that intend to enforce DMARC would need to verify the
DKIM signature of the mediator, and if the signature came from a
credible mediator and the tag is present, accept the message as though
it had an aligned signature.

-Jim

---

From: "Douglas E. Foster" <[email-masked]>
Date: Tue, 07 Jul 2020 00:56:39 GMT

This surprises me:

I expected that the tag would be a function of the algorithm used by the MLM or forwarder, rather than the particulars of each message.   In a face-to-face conversation between MLM and recipient email administrator, the algorithm would be the topic of conversation, and would be the determinant of whether trust was established or not.

At the same time, you raise an important point:   The tag will be most useful if it will be reliably correct, but less useful if it is prone to errors.  In practice, the tag is likely to be fraught with human errors which introduce another layer of trust confusion:   What do I do when the tag and the reversable content of the document are inconsistent with each other?  We have a lot of those problems already.

DF

---

From: "Douglas E. Foster" <[email-masked]>
Date: Tue, 07 Jul 2020 01:15:12 GMT

About discarding From alignment:
DMARC has been sold to big corporations as essential to defending their brand identity.    In response, they pay serious money to keep Valimail and its competitors in business.   I see no way that we can put forward a proposal that will put Valimail and its friends out of business, while incurring the wrath of C-Level executives and legal teams at Fortune 500 companies.   Not that I have any of those people on speed dial, so maybe someone can prove me wrong.   But I have been surprised that one of the DMARC reporting companies is not listening to this part of the discussion and having alarm bells.

About credible mediators:
You are right that if an inbound email gateway believes a Mediator is credible, then all that is necessary is for the Mediator to prove his own identity.    But what is the mechanism by which a Mediator obtains the trust of the email gateway?   And by what mechanism does the Mediator know that the email gateway has determined it to be credible?

One option is to provide so much information in the email message that the email gateway will grant trust on the fly.    Murray's approach has appeal because, especially when coupled with appropriate LIST-* and RESENT-* headers, it provides all of the information needed for a decision to be made.   ARC has less appeal because it provides just enough information for the email gateway to detect that something deliberate was done, but not much more.   Researching the credibility of the list would need to occur out-of-band using LIST-* headers as a starting point.   But there is a larger problem.   Even after the gateway decides the Mediator is credible, the Mediator is ignorant of the gateway's decision, so the Mediator is unable to take advantage of that decision.

The other option is for the MLM and the Email gateway administration to negotiate credibility in advance, with the subscriber acting as sponsor for the discussion.

DF

----------------------------------------
From: Jim Fenton <[email-masked]>
Sent: 7/6/20 2:33 PM
To: [email-masked]
Subject: Re: [dmarc-ietf] Fwd: New Version Notification for draft-kucherawy-dkim-transform-01.txt

-Jim

---

From: "John Levine" <[email-masked]>
Date: 6 Jul 2020 21:20:47 -0400

In article <[email-masked]> you write:
>I expected that the tag would be a function of the algorithm used by the MLM or forwarder, rather than the
>particulars of each message.

You would be mistaken. List management software has a vast array of
options for message handling. The changes to each message depend on
both how the particular list is configured and what's in the message.

R's,
John

---

From: "John Levine" <[email-masked]>
Date: 6 Jul 2020 21:30:16 -0400

In article <[email-masked]> you write:
>You are right that if an inbound email gateway believes a Mediator is credible, then all that is necessary is for
>the Mediator to prove his own identity.    But what is the mechanism by which a Mediator obtains the trust of the
>email gateway?   And by what mechanism does the Mediator know that the email gateway has determined it to be
>credible?

Please see previous discussions about the design of ARC. There's
reasons it's not just a whitelist of mailing lists.

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Mon, 6 Jul 2020 23:37:16 -0700

> valid message from some X, is the same.

There's a distinction though.  ARC tells you "that guy over there said the
original message passed", and you have to trust it.  On the other hand, the
transformations draft, when it works, hands you the original message, and
you don't have to make that trust assessment.

-MSK

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Mon, 6 Jul 2020 23:45:31 -0700

On Mon, Jul 6, 2020 at 10:12 AM Dave Crocker <[email-masked]> wrote:

> think the proposal has nothing at all to do with what the recipient sees.
> Rather, I've understood this as an attempt to reverse additions made by a
> Mediator, with the goal of validating the origination DKIM signature.
> Presumably that is so as to use the origination domain's reputation and
> even permit DMARC to validate.
>

Right, primarily.  It occurs to me that if you can recover the original
content and thus validate the author domain signature, you could opt to
deliver that version of the content to the user, permanently reversing the
transformations.  But that's mainly a reply to John's point that a spammer
could drop a blob of junk onto a legitimate message; the verifier could
simply reverse it.

-MSK

---

From: "Douglas E. Foster" <[email-masked]>
Date: Tue, 07 Jul 2020 11:37:27 GMT

I was convicted that I was working from a reading of the abstract, and not the whole document.   So I read the whole document last night, but it did not change my understanding.

Given a first occurrence of a message which purports to be from a legitimate message-altering MLM, how does the email gateway make a disposition decision?  ARC does not solve the problem, because the ARC chain does not provide enough information to do so.   ARC assumes that initial trust is obtained by another method.

Reversible tagging mostly solves the problem if the content is fully reversible.   The email gateway can assess both new and old content for risk, can assess both before and after signatures for identity verification, and can choose which version of content to deliver to the user.   But there are questions about whether this solution covers enough of the problem to make it the only solution.  We also have the question of whether the tagging would be too complex to do correctly.

For a complete solution, the mailing list needs to be able to distinguish between these possibilities:
Fiction - the mailing list does not exist, but a spammer wants to look like one to evade DMARC filteringUnsafe - the mailing list is poorly managed, so that it is little more than an open relay, or the content filtering is inadequate, so that it is a likely source of malicious content.Unwelcome - the subscriber address is a work account, and the list purpose is not work related.Spoofed list - a spammer tries to imitate a trusted list.Legitimate - a trusted and wanted list that both subscriber and email administrator want to receive.
I don't see that we have adequately addressed all of these scenarios, but again it seems that most of the issue hangs on the trust question.

List spoofing is an issue because we do not yet have a standard for how a list ID is associated with an authorized source.  Presumably, we should perform an SPF check on the list ID, or find a verified DKIM signature that is domain aligned with the list ID.   But there are no requirements on the format of a list ID, so we need to start by defining how the list domain is obtained from the list ID.

What is the process for obtaining trust (and a supporting exception in the email gateway)?
What is the process for notifying the MLM that the exception has been implemented?

DF

----------------------------------------
From: "John Levine" <[email-masked]>
Sent: 7/6/20 9:30 PM
To: [email-masked]
Cc: [email-masked]
Subject: Re: [dmarc-ietf] Fwd: New Version Notification for draft-kucherawy-dkim-transform-01.txt

---

From: Hector Santos <[email-masked]>
Date: Tue, 07 Jul 2020 08:47:46 -0400

> signature that they apply indicating that.

+1. By far, the simplest "ADD-ON" thing to do -- checking the MLM 
entry points.

> it had an aligned signature.

+1, this would work. The term "credible" will need to be defined as to 
how it is established.

Non-whitelisted local policy Authorized/Trusted resigners has been the 
central crux of the problem in the DKIM policy model. It needs to be 
automated and we can't get unfocused which the idea that "bad guys" 
can also be DKIM/DMARC compliant. Detected "Bad Guys" is a different 
dilemma.  I believe the of the top goals is to raise the bar for 
everyone so that we can get to that secondary Trust/Reputation layer. 
   The problem has been Signer Reputation wanting to push out DKIM Policy:

Layer 1: DKIM Policy Deterministic protocol
Layer 2: DKIM Signer Trust/Reputation

The argument has always been by Mr. Levine and others, why need Layer 
1 of all we need is layer 2?

The answer is that Layer 1 is an deterministic immediate filtering 
process to remove the contaminants of a mail stream, similar to an 
osmosis process. It includes checking RFC5322 correctness, like double 
5322.From lines, to address fake MUA from Display replay concerns.

But the push to use only Layer 1 perpetuated the brush back and 
delayed the adjusted needed in Layer 1 to address authorized resigner 
issues that Layer 2 will need.

If we go to back DKIM-STD, there was strong cogs efforts to remove or 
rather not recognized the Author Domain Identity (ADID) from a future 
DKIM Trust Assessment algorithm where only the SDID (Signer Domain 
Identity) and optional AUID (Agent User Identity) were parameters to a 
trust assessor.  I had argued (and lost) that the ADID needed to be 
part of the algorithm, officially in DKIM-STD.   But the effort was 
too strong to remove any recognition of its importance.  The cogs had 
a hard time doing this because DKIM policy was too strong of a 
concept. It would simply not go away. 5322.From could not be removed 
from DKIM-STD. It could not avoid the fundamental idea that 5322.From 
is the minimum required header to be bound to the signature.

---

From: Hector Santos <[email-masked]>
Date: Tue, 07 Jul 2020 09:09:09 -0400

On 7/7/2020 8:47 AM, Hector Santos wrote:

> 1 [if] all we need is layer 2?
>

Arggg, I really do need to take my time when I write.

Correction:

But the push to use only [Layer 2] perpetuated the brush back and
delayed the [adjustments] needed in Layer 1 to address authorized 
resigner
issues that Layer 2 will need.

---

From: "John R Levine" <[email-masked]>
Date: 7 Jul 2020 12:27:40 -0400

> you don't have to make that trust assessment.

I understand that, and I still don't see why it's useful.

It's hard to imagine a realistic sitatuation where a recipient system 
would strip off the changes and show the original message, so the 
recipient has to trust that the mediator doesn't make malicious 
transformations.  So if you trust them that far, why wouldn't you also 
trust them to report the status of incoming mail?

Regards,

---

From: Jim Fenton <[email-masked]>
Date: Tue, 7 Jul 2020 10:08:48 -0700

On 7/6/20 6:15 PM, Douglas E. Foster wrote:
> DMARC has been sold to big corporations as essential to defending
> their brand identity.    In response, they pay serious money to keep
> Valimail and its competitors in business.   I see no way that we can
> put forward a proposal that will put Valimail and its friends out of
> business, while incurring the wrath of C-Level executives and legal
> teams at Fortune 500 companies.   Not that I have any of those people
> on speed dial, so maybe someone can prove me wrong.   But I have been
> surprised that one of the DMARC reporting companies is not listening
> to this part of the discussion and having alarm bells.

While deployment is definitely a factor for IETF standardization
decisions, I would be very surprised to see IETF standardize a
specification that does not provide the benefits it claims to provide
just because it has been sold to big corporations. IETF doesn't need to
share in the wrath of those C-level executives.

Whether From alignment does or does not defend brand identity is a topic
for a different discussion thread. I was trying to avoid muddying this
discussion with it.

> You are right that if an inbound email gateway believes a Mediator is
> credible, then all that is necessary is for the Mediator to prove his
> own identity.    But what is the mechanism by which a Mediator obtains
> the trust of the email gateway?   And by what mechanism does the
> Mediator know that the email gateway has determined it to be credible?

My proposal is largely a simpler (I think) alternative to ARC for
assessing transformations. ARC also requires that the recipient's
verifier have some trust relationship with the mediator, perhaps through
an as-yet undefined reputation system.

Murray's reversible transforms proposal is yet another approach that
does not require trust to the same degree.

>
> One option is to provide so much information in the email message that
> the email gateway will grant trust on the fly.    Murray's approach
> has appeal because, especially when coupled with appropriate LIST-*
> and RESENT-* headers, it provides all of the information needed for a
> decision to be made.   ARC has less appeal because it provides just
> enough information for the email gateway to detect that something
> deliberate was done, but not much more.   Researching the credibility
> of the list would need to occur out-of-band using LIST-* headers as a
> starting point.   But there is a larger problem.   Even after the
> gateway decides the Mediator is credible, the Mediator is ignorant of
> the gateway's decision, so the Mediator is unable to take advantage of
> that decision.
>
> The other option is for the MLM and the Email gateway administration
> to negotiate credibility in advance, with the subscriber acting as
> sponsor for the discussion.

I'm not clear on which "gateway" you're referring to: the sender's
gateway? But "researching the credibility of the list" sounds like the
same problem as determining whether a mediator is credible, it's just
being done by a different party.

-Jim

---

From: Alessandro Vesely <[email-masked]>
Date: Tue, 7 Jul 2020 20:18:56 +0200

> I understand that, and I still don't see why it's useful.

It would allow me, for one, to honor remote DMARC policies.  Of course, I'd 
still need to manually whitelist non compliant MLMs.  However, when the number 
of those drops below a reasonable figure, whitelisting might become feasible.

> It's hard to imagine a realistic situation where a recipient system would 
> strip off the changes and show the original message, so the recipient has to 
> trust that the mediator doesn't make malicious transformations.

Agreed.  Undoing the changes has to be done on a temporary file, solely for 
verification purposes.  Undoing the changes would be illegal, if the footer 
contains legal claims.

> So if you trust them that far, why wouldn't you also trust them to report
> the status of incoming mail?

I cannot trust ARC operators, unless I manually compile a trusted list, which 
is as unfeasible as whitelisting each MLM.

I trust the original message as any other message.  Allowed transformations are 
designed to not pervert the original message.  Consider my notes 1 and 2 about 
l=, in a message upthread[*].  We can also specify limits on the size of 
subject tags and footers.  Transformations that insert stuff before the 
original content should not be allowed  —rewrite From: in such cases.

Best
Ale
-- 

[*] https://mailarchive.ietf.org/arch/msg/dmarc/oSX41FBGRGO_vP_4Qcd-rzIkWiI

---

From: Alessandro Vesely <[email-masked]>
Date: Wed, 8 Jul 2020 09:08:43 +0200

On Tue 07/Jul/2020 19:08:48 +0200 Jim Fenton wrote:
>> credible, then all that is necessary is for the Mediator to prove his own 
>> identity.    But what is the mechanism by which a Mediator obtains the trust 
>> of the email gateway?   And by what mechanism does the Mediator know that the 
>> email gateway has determined it to be credible?
> 
> My proposal is largely a simpler (I think) alternative to ARC for assessing 
> transformations. ARC also requires that the recipient's verifier have some 
> trust relationship with the mediator, perhaps through an as-yet undefined 
> reputation system.

The phrase /secret sauce/ is sometime used to refer to methods used to 
establish MTA reputation.  The need to keep ingredients secret is due to the 
ill-defined nature of the problem, whereby reputation can be gamed if its 
mechanisms are known.  That's why it is, and will be, as-yet undefined.

IOW, ARC is useful to a restricted number of large providers who are able to 
maintain a reliable reputation system.  Do they need a simpler alternative?

> Murray's reversible transforms proposal is yet another approach that does not 
> require trust to the same degree.

Right, a deterministic method.

Best
Ale
--

---

From: Dotzero <[email-masked]>
Date: Wed, 8 Jul 2020 04:21:15 -0400

> > I understand that, and I still don't see why it's useful.
At what overhead cost? You have to hold the connection open while reversing
the transformations or you are not in a position to reject (vs accepting
then rejecting). There are folks currently holding the connection while
evaluating the DKIM signature but that is lighter weight than reversing the
transforms AND doing the DNS lookup to validate the DKIM signature.

> of those drops below a reasonable figure, whitelisting might become
An interesting dependency. When might that reasonable figure occur? What is
a reasonable number? Would other receivers agree with you on
reasonableness? Should a standard be predicated on this basis?

> verification purposes.  Undoing the changes would be illegal, if the

Your claim is interesting. What possible legal claims might an
intermediator make that would be legally binding on a receiver and be
illegal to undo in a temporary file for processing? Are you a lawyer? Are
you a lawyer making such an assertion for all jurisdictions?

> is as unfeasible as whitelisting each MLM.

Or you use a trusted list compiled by someone else. The use of blocklists
such as Spamhaus ones or RPZ  come to mind.

> original content should not be allowed  —rewrite From: in such cases.

This is getting unduly complicated. Now we have to hold up progress on the
effort while folks argue out what the size limits on subject lines and
footers should be?

Michael Hammer

---

From: Alessandro Vesely <[email-masked]>
Date: Wed, 8 Jul 2020 12:15:59 +0200

> transforms AND doing the DNS lookup to validate the DKIM signature.

In principle, reversing the transformation shouldn't be more complicated than 
verifying a signature with l=.  Skipping the subject tag while canonicalizing 
the header is not much more work than adding \r to each line (at mine, lines 
are terminated by a bare \n.)  The addition of an entity may require to skip an 
initial part of the body, and then stop at the corresponding boundary.  It can 
be considered a refined canonicalization, negligible w.r.t. cryptography.

>> It would allow me, for one, to honor remote DMARC policies.  Of course, 
>> I'd still need to manually whitelist non compliant MLMs.  However, when
>> the number of those drops below a reasonable figure, whitelisting might
>> become feasible. >>
>>

Nowadays, I honor remote DMARC policies on a few domains only, (paypal, amazon, 
and the like.)  I cannot switch to the opposite behavior, to honor DMARC 
globally except for a few exceptions, because From: rewriting is considered 
dirty and there is no other way for MLMs which need to have a footer.  IOW, 
DMARC is broken.

If we predicate a standard which makes it possible for MLMs to comply, then 
honoring DMARC becomes possible.  MLMs which don't comply can result in missed 
messages until receiving MXes whitelist them.  Those glitches are justified 
because the culprit is the MLM in that case, not the MX.  When there are enough 
examples of working mailing lists, that kind of explanation may sound 
acceptable to users.

>>> It's hard to imagine a realistic situation where a recipient system
>>> transformations. >>
>> Agreed.  Undoing the changes has to be done on a temporary file, solely 
>> for verification purposes.  Undoing the changes would be illegal, if the 
>> footer contains legal claims. >>
> 

IANAL, but I heard that argument by MLMs holding they cannot just omit any 
changes because blah blah.  There are also forwarders who want to add 
disclaimers or spam reporting directives as a footer.  Tampering with the mail 
they send sounds likely to be an offense in a number of countries.  Of course, 
the same allegation can be made about the transformations they applied in the 
first place.  However, MLMs should have managed to get subscribers consensus to 
do so.

To endow a message with instruction to revert transformations should not imply 
an authorization to deliver the message with those transformations removed. 
The deal should be to supply that data for signature verification *only*.

>>> So if you trust them that far, why wouldn't you also trust them to
>>> report the status of incoming mail? >>
>> I cannot trust ARC operators, unless I manually compile a trusted list, 
>> which is as unfeasible as whitelisting each MLM. >>
> 

Yeah, I use both those blocklists.  I also use dnswl, which actually can be 
configured as a filter to prevent applying DMARC policies.  I only use it to 
prevent SPF reject-on-fail.  The difference between SPF and DMARC is that the 
former has other means to do the right thing, so that dnswl works as a second 
level savior.

The standard should describe a few (deterministic) ways to be DMARC compliant. 
  At the moment we only have From: rewriting, which is not even specified.  I 
proposed to use To: instead of From:, which would be interesting to experiment. 
  Murray's transformations is yet another way, particularly nice as it would 
allow mailing lists like this one to resume working the way they used to.

>> I trust the original message as any other message.  Allowed 
>> transformations are designed to not pervert the original message.
>> Consider my notes 1 and 2 about l=, in a message upthread[*].  We can also
>> specify limits on the size of subject tags and footers.  Transformations
>> that insert stuff before the original content should not be allowed
>> —rewrite From: in such cases. >>
> 

It should not be allowed to insert text long enough (including trailing spaces) 
to push the original subject out of view.  I don't think determining this 
number is the hardest part of the spec.

Best
Ale
--

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Wed, 8 Jul 2020 09:04:28 -0700

On Wed, Jul 8, 2020 at 1:21 AM Dotzero <[email-masked]> wrote:

> At what overhead cost? You have to hold the connection open while
> reversing the transformations or you are not in a position to reject (vs
> accepting then rejecting). There are folks currently holding the connection
> while evaluating the DKIM signature but that is lighter weight than
> reversing the transforms AND doing the DNS lookup to validate the DKIM
> signature.
>

That seems to imply DKIM, which relies on DNS and cryptography, is more
heavyweight than reversing text transformations, which relies only on the
local CPU and memory and probably arithmetic.  I'm not sure I agree.

-MSK

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Wed, 8 Jul 2020 09:42:44 -0700

I'm sorry, I got that backwards.  What I meant to say is:

>> reversing the transforms AND doing the DNS lookup to validate the DKIM
That seems to imply DKIM, which relies on DNS and cryptography, is LESS
heavyweight than reversing text transformations, which relies only on the
local CPU and memory and probably arithmetic.  I'm not sure I agree.

-MSK

---

From: Dotzero <[email-masked]>
Date: Wed, 8 Jul 2020 16:08:45 -0400

> local CPU and memory and probably arithmetic.  I'm not sure I agree.
Once you reverse the transformations you would still need to do the DKIM
lookup to validate the reversed text that was signed. Simply reversing the
transformations without validating doesn't give you much of anything useful.

Michael Hammer.

---

From: Jim Fenton <[email-masked]>
Date: Wed, 8 Jul 2020 15:37:00 -0700

On 7/8/20 1:08 PM, Dotzero wrote:
>
> Once you reverse the transformations you would still need to do the
> DKIM lookup to validate the reversed text that was signed. Simply
> reversing the transformations without validating doesn't give you much
> of anything useful.

DKIM lookup: do you mean the key retrieval? If so, that can be done in
parallel with any of the transformation stuff, which is all local
computation.

-Jim

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Wed, 8 Jul 2020 18:03:40 -0700

> transformations without validating doesn't give you much of anything useful.

Of course, but both DKIM signatures will be validated with or without this
proposal, which requires processing the entire message body anyway.

A sub-optimal flow might be:

- validate the MLM signature; it passes
- validate the author domain signature; it fails
- notice the MLM signature (which validated) has a "tf=" tag saying
"subject,footer"
- search the body backwards until you find the delimiter, and toss away
everything after that, which undoes the "footer"
- un-mutate the Subject
- re-validate the author domain signature (you don't have to repeat the DNS
stuff, it's cached); it (in theory) passes

<hand_waving> You could maybe do clever tricks like notice that "tf" is
there and apply the reverses inline as you process the author signature,
saving you a repeat verification pass.  </hand_waving>

In any case, now you know the same thing ARC told you, but you didn't have
to do more crypto than just the two DKIM signatures than you were going to
do anyway, and you avoid larding the header as ARC does.

I have less confidence in the expense of the MIME un-transformations
because I've never implemented them specifically.  I could take a run at it
but there are probably libraries out there that do a better job than I
would.

-MSK

---

From: Alessandro Vesely <[email-masked]>
Date: Thu, 9 Jul 2020 11:50:29 +0200

On Thu 09/Jul/2020 03:03:40 +0200 Murray S. Kucherawy wrote:
> - notice the MLM signature (which validated) has a "tf=" tag saying "subject,footer"
> - search the body backwards until you find the delimiter, and toss away everything after that, which undoes the "footer"
> - un-mutate the Subject
> - re-validate the author domain signature (you don't have to repeat the DNS stuff, it's cached); it (in theory) passes
> 
> [...ARC comparison omitted...]
> 

It'd be a stroke of luck to find a library which does exactly that.  Ned's note 
about the design of such tools applies here too.  However, it doesn't seem 
overly difficult to do it from scratch.

Let's consider a sample multipart message body, indented for legibility:

     --added-boundary
     Content-Type: <original-content-type>; boundary="original-boundary"

         [...original preamble...]
         --original-boundary
         [...original header and body...]
         --original-boundary--
         [...original epilogue...]

     --added-boundary
     [...added footer...]
     --added-boundary--
     [...epilogue...]

Assume for simplicity that the author did not sign Content-Type:.  Knowing 
there's an added boundary, you can skip the initial part of the body, start 
canonicalizing at the first entity's body, and stop at the next appearance of 
"--added-boundary".  In addition, you must also check that there are no 
entities before the original part.  So, all you need to know is that the 
original message was multipart.

MLM code probably needs some rewriting in order to ensure that the original 
preamble and epilogue are kept intact (and without adding extra empty lines.) 
Some rewriting is also needed to avoid gratuitous header changes.  I tried the 
above procedure manually on the message I'm replying to.  It failed because the 
original message has:

     Cc: Alessandro Vesely <[email-masked]>, IETF DMARC WG <[email-masked]>

whereas the published copy has:

     Cc: IETF DMARC WG <[email-masked]>, Alessandro Vesely <[email-masked]>

For text/plain messages, instead, all you need to know is the length of the 
original message.  If the author signed with l=, no action is needed.

In both cases, the body hash can be done in one pass, except if the body is 
converted to base64 encoding.  This list often, but not always, does such 
wrapping, and then signs the message again.  Dunno if that's a feature or a bug.

Best
Ale
--