Subject: [Tools-discuss] emails being truncated

From: "STARK, BARBARA H" <[email-masked]>
Date: Fri, 16 Jul 2021 14:50:51 +0000

My emails (to [email-masked]) are suddenly being strangely truncated. I've never seen this before. Is anyone else experiencing this, or experienced this in the past?
Barbara

---

From: Robert Sparks <[email-masked]>
Date: Fri, 16 Jul 2021 10:54:04 -0500

Hi Barbara -

Your messages look intact to me both through imap.ietf.org and using 
https://mailarchive.ietf.org/arch/browse/dnssd/

If that looks right to you, perhaps you could forward the raw message 
you got from the list that didn't look right to both me and Ryan (copied)?

RjS

On 7/16/21 9:50 AM, STARK, BARBARA H wrote:
> ___________________________________________________________
> Tools-discuss mailing list - [email-masked]
> This list is for discussion, not for action requests or bug reports.
> * Report datatracker and mailarchive bugs to: [email-masked]
> * Report tools.ietf.org bugs to: [email-masked]
> * Report all other bugs or issues to: [email-masked]
> List info (including how to Unsubscribe): https://www.ietf.org/mailman/listinfo/tools-discuss

---

From: Toerless Eckert <[email-masked]>
Date: Fri, 16 Jul 2021 18:11:05 +0200

Do you have an example date / message id of such an email ?

Does the dnssd email archive have the email untruncated but only group members
report that it was truncated on their end, or is it also truncated on the
list archive ?

I had once email recently from an AD via draft-alias and WG expander,
which arrived truncated on my side, but made it untruncated to
the WG alias.

Alas, i have not been able to track thast one down, even
through it was a mayor issue for me (AD review of one of my drafts
that i received truncated. Imagine how i felt when the AD told me:
why did you stop fixing nits after 25% of my review... ;-))

Cheers
    Toerless

-- 
---
[email-masked]

---

From: Toerless Eckert <[email-masked]>
Date: Mon, 19 Jul 2021 17:08:24 +0200

What is actually the right place to raise email error issues ?

I just received the following email error back
for email to [email-masked]:

This is the mail system at host ietfa.amsl.com.                                                                                                                                                                       
I'm sorry to have to inform you that your message could not                                                
be delivered to one or more recipients. It's attached below.                                               
                                                                                                           
For further assistance, please send mail to postmaster.                                                    
                                                                                                           
If you do so, please include this problem report. You can                                                  
delete your own text from the attached returned message.                                                   
                                                                                                           
                   The mail system                                                                         
                                                                                                           
<[email-masked]>: Command died with status 1:                                             "/a/www/ietf-datatracker/web/ietf/virtualenv-manage.py feedback_email                                      --nomcom-year 2021". Command output: CommandError: 'utf-8' codec can't                                     decode byte 0xb7 in position 114: invalid start byte

Cheers
    Toerless

-- 
---
[email-masked]

---

From: Robert Sparks <[email-masked]>
Date: Mon, 19 Jul 2021 10:17:31 -0500

[email-masked] would be appropriate for this kind of failure.

I've forwarded this one there already, and will follow up with you 
individually.

RjS

---

From: "STARK, BARBARA H" <[email-masked]>
Date: Wed, 21 Jul 2021 15:33:35 +0000

Following up for those who may be curious...
I did email the support team with copies of the sent and received versions of emails. Glen correctly diagnosed the problem.
My email contained a line with exactly 76 characters, with last character ".", and followed by carriage return. Apparently, my Exchange server wrapped this line at 75 characters to put that "." all by itself on a line (with the carriage return).
The emails I received were being truncated directly before that ".".
This is a known bizarre issue.
https://unix.stackexchange.com/questions/38156/sendmail-procmail-exchange-truncating-mail

The answer there explains that
" - Exchange: on accepting a piece of mail for delivery, it appears to reformat a plain text message, encoding and wrapping its lines to 75 characters.
   - sendmail: An old (but known) behaviour was being followed in that mail with a bare period on a line was interpreted as end-of-message and then delivered, effectively truncating the actual mail body."

Well, I feel better now.

Barbara

> > My emails (to [email-masked]) are suddenly being strangely truncated. I've
> never seen this before. Is anyone else experiencing this, or experienced this in
> the past?
> > Barbara
> >
> > List info (including how to Unsubscribe):
> https://urldefense.com/v3/__https://www.ietf.org/mailman/listinfo/tools-
> discuss__;!!BhdT!y69XSrVx9_N0CqsoYvkdc4xaLGtXfH6kh-
> qvbmUhVIgNfw0yBw6GXgb2ZApVtA$
> 
> --
> ---
> [email-masked]

---

From: 'Toerless Eckert' <[email-masked]>
Date: Wed, 21 Jul 2021 18:55:27 +0200

Thanks, Barbara, but i can still not make out heds or tails
(adding Glen and John)

a) Was the bug now fixed ? Aka: when you repeat this, will it work now ?

b) Whether fixed or not, which piece of software is the culprit ?

AFAIK:

The "old" behavior you refer to is the standard SMTP end-of-mail-data
"<CRLF>.<CRLF>" behavior of 1982 RFC821 section 3.1  that must go
along with the transparency procedure of section 4.5.2. Current
SMTP RFC5321 does not change this behavior.

So, you can see why i am curious about any software having a bug
about a procedure that has been used in gazillions (too many to count)
of emails since 1982.

The way you describe it sounds as if an Exchane server must be
speaking SMTP, and it is folding long lines AFTER performing the
SMTP transparency fixup, which is the wrong order. 

The only SMTP server/message-receiver side issue i can think of is
confusion introduced when going beyond ASCII about what constitutes
a ".". RFC5321 hints at this, but does not explain the breaking workflow.

Cheers
    Toerless

-- 
---
[email-masked]

---

From: "STARK, BARBARA H" <[email-masked]>
Date: Wed, 21 Jul 2021 17:29:54 +0000

Hi Toerless,
I'm fairly sure the problem lies within the AT&T email systems, which are very convoluted (for security reasons). I don't think this is a problem that can be solve by IETF. I'm not particularly incented to try to fix the AT&T email issue because such things always involve going through a Tier 1 "help" desk that will want to invasively take control of my laptop to try to "fix" my problem (because all problems are on users' local machines, of course). They may break something that works, while doing this. I would then have to convince this person half a world a way to please escalate. It's not worth it to me, so I'm going to leave it here.
Barbara
 
> > I did email the support team with copies of the sent and received versions of
> emails. Glen correctly diagnosed the problem.
> > My email contained a line with exactly 76 characters, with last character ".",
> and followed by carriage return. Apparently, my Exchange server wrapped this
> line at 75 characters to put that "." all by itself on a line (with the carriage
> return).
> https://urldefense.com/v3/__https://unix.stackexchange.com/questions/38156
> /sendmail-procmail-exchange-truncating-
> mail__;!!BhdT!y4NKbaHihIlZDD_qjVaDSgFEXbVqZpRI00Bnmb63dTsUcMAY6vSUA
> QCQI5jkaA$
> >
> > " - Exchange: on accepting a piece of mail for delivery, it appears to reformat
> a plain text message, encoding and wrapping its lines to 75 characters.
> >    - sendmail: An old (but known) behaviour was being followed in that mail
> with a bare period on a line was interpreted as end-of-message and then
> delivered, effectively truncating the actual mail body."
> >
> > Well, I feel better now.
> >
> > Barbara
> >

---

From: 'Toerless Eckert' <[email-masked]>
Date: Wed, 21 Jul 2021 20:41:01 +0200

Let me add emailcore

On Wed, Jul 21, 2021 at 05:29:54PM +0000, STARK, BARBARA H wrote:
> Hi Toerless,
> I'm fairly sure the problem lies within the AT&T email systems, which are very convoluted (for security reasons).

Thanks.

> I don't think this is a problem that can be solve by IETF.

Let me disagree!

It is of course very frustrating to see such bugs in deployments
for a spec from 1982 (RFC821/RFC5321). But if the problem does happen in
a large enterprise as ATT, it likely happens elsewhere too,
and after 43 years its clear it will not go away by itself:

I suspect that after 43 years this is not even bad old code, but
badly written "recent" SMTP code. SMTP sending looks from the outset so
simple, many attempt to just re-implement it as few lines of
code in their app and of course miss out on details. I know
i did (statue of limitations for me ;-).

emailcore is working on RFC5321bis. 

Hardening protocol state machineries against broken peers
is IMHO extremely important, for sond principles such as
"be lenient in what you accept and diligent in what you send",
or providing good error diagnostics.

This particular part of the SMTP state machinery is one which
doesn't do either. In my memory almost the only.

I do suspect ATTs broken sender would continue to send
the message body after the ".", but the SMTP peer will
just interpret that as broken SMTP. If that is the case,
it should be possible to use an expanded SMTP state machinery
with such an error recognition, even if just heuristic.

And heuristic is good enough. It just takes few correctly
triggeed error email notifications from the heuristic to get bugs
like this fixed over the years, especially when the error email
generated was easy to understand.

I would be happy to suggest a heuristic if emailcore was
willing to think about adding this as an item to fix
for draft-ietf-emailcore-rfc5321bis.

Worst case, you might need to send more emails with
the problematic 76 characters to test the theory ;-)

Cheers
    Toerless

> I'm not particularly incented to try to fix the AT&T email issue because such things always involve going through a Tier 1 "help" desk that will want to invasively take control of my laptop to try to "fix" my problem (because all problems are on users' local machines, of course). They may break something that works, while doing this. I would then have to convince this person half a world a way to please escalate. It's not worth it to me, so I'm going to leave it here.
> Barbara
>  

-- 
---
[email-masked]

---

From: "John Levine" <[email-masked]>
Date: 21 Jul 2021 17:19:00 -0400

>a) Was the bug now fixed ? Aka: when you repeat this, will it work now ?

As she said, it's a well known bug in Microsoft Exchange.

>b) Whether fixed or not, which piece of software is the culprit ?

It's still in Microsoft Exchange, which is famous for its casual relationship
to mail standards.

R's,
John

---

From: "John Levine" <[email-masked]>
Date: 21 Jul 2021 17:21:21 -0400

>i did (statue of limitations for me ;-).

An interesting guess, but clearly wrong, since it's still a well known bug
in Microsoft Exchange.

How about if you pop up to Redmond, explain to Microsoft what they're doing
wrong, and let us know what they say?

R's,
John

---

From: John C Klensin <[email-masked]>
Date: Thu, 22 Jul 2021 09:15:02 -0400

--On Wednesday, July 21, 2021 18:55 +0200 'Toerless Eckert'
<[email-masked]> wrote:

>...
> The only SMTP server/message-receiver side issue i can think
> of is confusion introduced when going beyond ASCII about what
> constitutes a ".". RFC5321 hints at this, but does not explain
> the breaking workflow.

In what way do you consider the text in RFC6321 a "hint"?  Is it
because the text refers to "period" and, in several places, to
"<CRLF>.<CRFL>" without spelling out that the period is ASCII
(2/14 in the original notation, Hex 2E)?    Are the restrictions
to ASCII in Section 2.3.1 insufficient?

Because RFC 5321 is being revised in the EMAILCORE WG (where you
are welcome to bring this up if you think it is important), it
is quite easy to add the hex value to the "(period or full
stop)" note  in Section 3.3.  Would that be sufficient to make
the text more than a hint, or it just being ridiculously
pedantic?

   john