Subject: [marf] reporting-discovery

From: "J.D. Falk" <[email-masked]>
Date: Fri, 14 Jan 2011 15:12:42 -0800

I've uploaded an updated version of the discovery draft:

http://tools.ietf.org/html/draft-jdfalk-marf-reporting-discovery-03

Notable changes since version 02:

- Removed references to end-user report submissions; those should be left for a separate document, and probably be discoverable in the same way as MUA settings.

- Rewrote some confusing parts.

- Added the examples section.

Remaining questions:

- should this include IODEF as a format?

- should the advertisement be published somewhere other than DNS?

- should this document include a description of how the complaint feedback signup process works today, and how this proposal would change it?  Or is that a separate Informational document?

- should the current Privacy Considerations section be moved to a separate BCP?

- why does the submission tool keep thinking my name is "R. Path?"

And, most importantly: is there consensus to make this a WG document?  I'm willing to continue as editor.

---

From: Alessandro Vesely <[email-masked]>
Date: Sun, 16 Jan 2011 14:59:35 +0100

> - should this include IODEF as a format?

I'd say this would be a Good Thing (TM).  However, it is them who
should ask for it, and define some details.  Perhaps, we should prompt
them to do so? (I add a CC, in case.)  And what about x-arf?

A different report format may imply different transports, thus the I-D
may need to specify that r= email addresses are only meant for email
transport.  More importantly, report types may differ:  It is highly
desirable that report types use the same words to mean the same type,
across any format.  Otherwise, the rt should be appended to each rf,
e.g. as "rf=ARF:abuse,fraud; rf=IODEF:thraud;".

> And, most importantly: is there consensus to make this a WG document?

I, for 1, vote for this I-D to become a WG doc.  Let me recall it is
due by Sep 2011, according to the charter.

A few more mumblings, in no particular order:

* [MARF-BASE] is now RFC 5965.

* We probably should devise a tool that works according to this I-D.
(Maybe a command line Perl script that forwards complaints to the
relevant ADMD...)

* In sections 5.1 and 5.2 a phrase like "The following tags are
defined for (Consumers/Generators)" before the relevant tags list,
would improve readability.

* Shouldn't the default r= be abuse@, rather than postmaster@?

* Saying that the default policy is rp=o obviously implies that
sending complaints to the default address is fine.  Should the I-D
explicitly say that those defaults may be assumed even when no DNS
record (or whatever) is defined at all?

---

From: Shmuel (Seymour J.) Metz <[email-masked]>
Date: Mon, 17 Jan 2011 09:34:44 -0500

In <[email-masked]>, on
01/14/2011
   at 03:12 PM, "J.D. Falk" <[email-masked]> said:

>I'm willing to continue as editor.

It would seem to be in scope.

-- 
     Shmuel (Seymour J.) Metz, SysProg and JOAT
     Atid/2        <http://patriot.net/~shmuel>
We don't care. We don't have to care, we're Congress.
(S877: The Shut up and Eat Your spam act of 2003)

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Tue, 18 Jan 2011 09:38:16 -0800

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On Behalf Of Shmuel Metz
> Sent: Monday, January 17, 2011 6:35 AM
> To: [email-masked]
> Subject: Re: [marf] reporting-discovery
> 

[as participant]

+1, and it looks to be a good place from which to start this part of our chartered work.

---

From: "J.D. Falk" <[email-masked]>
Date: Tue, 18 Jan 2011 11:46:24 -0800

> them to do so? (I add a CC, in case.)  And what about x-arf?

The X-ARF authors don't appear to be interested in rejoining the IETF process, so there's no I-D to refer to.

> e.g. as "rf=ARF:abuse,fraud; rf=IODEF:thraud;".

Yep, that's what's stopped me from adding IODEF already.  Perhaps someone more familiar with the format could help us figure out how/if to add it?

> (Maybe a command line Perl script that forwards complaints to the

Agreed.

> * Shouldn't the default r= be abuse@, rather than postmaster@?

Maybe.  I chose postmaster@ because it's required (MUST) for operating a mail system, but abuse@ may be sufficiently standard by now.  Any thoughts from the rest of the group?

> record (or whatever) is defined at all?

That would impose a requirement that all mail operators accept and process ARF reports, even if they haven't chosen to participate in any ARF-related standard.  In other words: they'll start getting messages this weird MIME type they've probably never heard of before, and they'll probably be annoyed.

(Involuntary ARF reporting is also discussed in one of the appendices of draft-jdfalk-maawg-cfblbcp.)

---

From: Tony Hansen <[email-masked]>
Date: Tue, 18 Jan 2011 14:53:25 -0500

> Maybe.  I chose postmaster@ because it's required (MUST) for operating a mail system, but abuse@ may be sufficiently standard by now.  Any thoughts from the rest of the group?

+1 for abuse@

     Tony Hansen

---

From: Steve Atkins <[email-masked]>
Date: Tue, 18 Jan 2011 12:01:29 -0800

> Maybe.  I chose postmaster@ because it's required (MUST) for operating a mail system, but abuse@ may be sufficiently standard by now.  Any thoughts from the rest of the group?

re=, I'm guessing, not r=?

If so, +1 for abuse@. 

Cheers,
  Steve

---

From: Tobias Knecht <[email-masked]>
Date: Tue, 18 Jan 2011 21:23:56 +0100

Hello everybody,

> process, so there's no I-D to refer to.

Sorry for the confused question? Which IETF process? The ARF one or the
process about reporting-discovery?

At the moment we do not have too much to say about the ARF process. We
are using ARF for spam reports and that's what it's made for. We are
using x-arf for everything else. Unfortunately there was not much
movement in the development of x-arf in the last weeks, but this will
change soon.

Again, if there is somebody interested in xarf. Having ideas, wants to
implement it, let me know and we can subscribe you to our mailinglist.

About the reporting-discovery draft: I like the idea, even though it is
not fitting in our thinking of "global reporting". Never the less I
think it would be nice for x-arf being part of it.

Thanks,

Tobias

-- 
abusix

---

From: John Levine <[email-masked]>
Date: 19 Jan 2011 03:56:06 -0000

>Maybe.  I chose postmaster@ because it's required (MUST) for
>operating a mail system, but abuse@ may be sufficiently standard by
>now.  Any thoughts from the rest of the group?

Mr. Abuse.net says that default to postmaster doesn't work very well.
Why not make the full e-mail address mandatory?  If you're going to go
to the trouble to publish a _report record, it's really not an undue
burden to require that you tell people where the reports should go.

I sent some notes about the security section, so you can consider them
included by reference here.

R's,
John

---

From: Barry Leiba <[email-masked]>
Date: Tue, 18 Jan 2011 23:23:37 -0500

> burden to require that you tell people where the reports should go.

I agree.  There doesn't need to be a default.

Barry, as participant

---

From: Shmuel (Seymour J.) Metz <[email-masked]>
Date: Wed, 19 Jan 2011 10:56:29 -0500

In <[email-masked]>, on
01/18/2011

>now.  Any thoughts from the rest of the group?

While RFC 2142 is only informational, I believe that it is appropriate
to use it as the basis for the default. Since the default in
draft-jdfalk-marf-reporting-discovery-03.txt only applies after the RR
is added, I would expect any e-mail admin without an abuse account to
explicitly specify the mailbox that he wants the MARF reports sent to.

---

From: "J.D. Falk" <[email-masked]>
Date: Mon, 24 Jan 2011 17:19:32 -0800

> In <[email-masked]>, on

I think I'm seeing about equal consensus for making abuse@ the default, or for having no default and requiring an address.  

Personally, I'm leaning towards having a default in order to keep the record small.

Any other opinions?

And, any additional thoughts on whether to pull this document into the WG?

---

From: "J.D. Falk" <[email-masked]>
Date: Mon, 24 Jan 2011 17:24:13 -0800

> process about reporting-discovery?

There's a process for extending ARF, but X-ARF hasn't followed that process, so I'd assumed y'all just weren't interested in making it an open standard.

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Mon, 24 Jan 2011 17:24:52 -0800

> From: [email-masked] [mailto:[email-masked]] On Behalf Of J.D. Falk
> Sent: Monday, January 24, 2011 5:20 PM
> To: Message Abuse Report Format working group
> I think I'm seeing about equal consensus for making abuse@ the default,
> or for having no default and requiring an address.
> 

[as participant]

I'm leaning toward having the default.  In the absence of this record, that's what most people will do anyway since there's already an RFC that encourages it.

---

From: Steve Atkins <[email-masked]>
Date: Mon, 24 Jan 2011 17:29:25 -0800

> Personally, I'm leaning towards having a default in order to keep the record small.

Works for me - I don't have a strong preference either way, but the record size issue does tilt it slightly towards using a default of abuse@.

> And, any additional thoughts on whether to pull this document into the WG?

I think that would be a good thing.

Cheers,
  Steve

---

From: Barry Leiba <[email-masked]>
Date: Mon, 24 Jan 2011 21:29:34 -0500

> I think I'm seeing about equal consensus for making abuse@ the default, or for
> having no default and requiring an address.
>

I favour requiring it because it makes them think about it, and
perhaps make sure the specified address really exists and is actively
handling messages.  Something left to a default can easily be
non-existant, or a black hole.

That said, I don't feel strongly, and am happy to agree to whichever
winds up in the document.

Barry (participant)

---

From: John Levine <[email-masked]>
Date: 25 Jan 2011 03:08:18 -0000

>winds up in the document.

Agreed.  I don't see the record size as a big deal.

R's,
John

---

From: Alessandro Vesely <[email-masked]>
Date: Tue, 25 Jan 2011 13:01:57 +0100

On 25/Jan/11 02:24, Murray S. Kucherawy wrote:
> I'm leaning toward having the default.  In the absence of this
> record, that's what most people will do anyway since there's
> already an RFC that encourages it.

Also for "r"?  (See Appendix C of draft-jdfalk-maawg-cfblbcp)

I wonder if it would make sense to note these concerns explicitly in
the I-D, or to require a generator record when assuming default values
from non-found consumer records; e.g. something like

  5.1.2 Feedback Consumer default

  As the above provisions imply, mail senders who publish no Feedback
  Reporting advertisement may happen to receive reports by generators
  who assume the default values.  That is, generators MAY send ARF
  reports to abuse@ addresses unless different dispositions exist.
  While this is consistent with RFC 2142, it has to be noted that a
  default address can easily be non-existant, or a black hole.  Thus,
  consumers are encouraged to publish explicit information even in
  case it happens to agree with the default values:  in addition to
  better clarity, it may also improve caching.

  Generators who send reports to abuse@ in the absence of any
  explicit advertisement record SHOULD publish a generator record as
  detailed in the following section.

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Fri, 28 Jan 2011 11:16:26 -0800

> Sent: Friday, January 14, 2011 3:13 PM
> Subject: [marf] reporting-discovery
> 
> [...]

[as co-chair]

Since we have a chartered item to produce a reporting discovery proposal and this is the only draft submitted thus far, and it sparked some discussion, I believe it's appropriate that we make it into a working group item.

If there's no objection, I'll do so next week.  If you have an issue with it, please say so soon.

-MSK

---

From: <[email-masked]>
Date: Sun, 30 Jan 2011 19:13:30 -0500

Sorry for my delay in response.  I should be able to look closer at this later in the week.  I am copying Brian who helped in the IODEF work as well in case he can get to this sooner than I can.

The basic question being asked is to know if reports could be provided in an IODEF format, right?  This should be fine and I think it would be a good idea to allow for that option as IODEF covers all types of incidents.  If there are some things not covered in IODEF, an extension may be necessary and the specification allows for that.

What do you need from us in order to add this option?

Thank you,
Kathleen

-----Original Message-----
From: J.D. Falk [mailto:[email-masked]] 
Sent: Tuesday, January 18, 2011 2:46 PM
To: Alessandro Vesely
Cc: [email-masked]; Moriarty, Kathleen
Subject: Re: [marf] reporting-discovery

Agreed.

---

From: Alessandro Vesely <[email-masked]>
Date: Mon, 31 Jan 2011 14:42:47 +0100

Hi Kathleen,
thank you for participating!

On 31/Jan/11 01:13, [email-masked] wrote:
> The basic question being asked is to know if reports could be
> provided in an IODEF format, right?  This should be fine and I
> think it would be a good idea to allow for that option as IODEF
> covers all types of incidents.  If there are some things not
> covered in IODEF, an extension may be necessary and the
> specification allows for that.

For some background, the (Mail) Abuse Report Format, as its name
suggests, is primarily used to report abuse, possibly in relation with
feedback loops [FBL].  In some situations, abuse reports could be sent
to a domain's abuse@ address without prior agreement, or they may
happen to reach some staff mailbox somewhat unexpectedly.  For this
reason, the format has been designed to be as human readable and self
explanatory as possible.

Mail abuse reports may be initiated by end users clicking a
This-is-Spam button on their MUAs, and thence reach FBL subscribers
--a scenario currently limited to a few huge North-American mailbox
providers.  It is not yet well defined how these reports will, in
general, be forwarded, to what addresses, using what format, and what
transport.  I look forward to further insight from you on this subject.

At this early stage, we cannot know in what cases which format
conversions may be necessary or advisable in order to forward reports
adequately.  The idea is that an MTA would look up some kind of
reputation service, internal policy, or FBL subscription file in order
to determine what ADMDs, if any, deserve a copy of ARs received from
end users.  Looking up the relevant _report RR in the DNS would then
provide the exact address and format to use.

> What do you need from us in order to add this option?

Four report types [ARF#T] have been standardized for ARF:

   abuse, fraud, virus, and other.

The reporting discovery [RD] resource record definition provides for
both feedback consumers and generators to be able to specify the type
of reports that they are willing to receive or send, respectively.
The syntax of RR specifies the tags rt= and gt=, whose values should
be one or more of those tokens, as defined in the [IANA-MARF].

IODEF tokens, instead, are registered in the framework of XML
namespaces and schemas, as for example, [IANA-PHISH].  Does this
extension cover all email-related IODEF incidents?  The phishing
extension [PHISH] defines the following fraud types:

   phishing, recruiting, malware distribution, fraudulent site,
   dnsspoof, archive, other, unknown, and ext-value.

The type can be a key information for deciding what route a report
should be forwarded to, or which team will deal with it.  How are we
going to map these types across format conversions?  Or should the
type be replaced with some other token or omitted altogether?

IODEF and MARF target different users.  The global number of network
providers seems to be smaller than that of mailbox providers, so that
the formers may be expected to have better means to learn the
addresses of one another than looking up the DNS.  OTOH, a stream of
generic IODEF reports may be of interest to generic firewall
operators, even if they are not actually network providers.  Would it
be helpful for a network provider to offer such stream to interested
consumers using the DNS?  (IOW, why didn't you define the analogous of
reporting discovery?)

Non-cooperative mail senders, including bots, deserve being reported
to some upstream network provider.  In case the provider manages the
relevant DNS zone, it may publish reporting information.  This
includes format, type, and address preferences.  Are these orthogonal
coordinates, or should the RR syntax provide for a different arrangement?

-- 
[FBL] http://en.wikipedia.org/wiki/Feedback_Loop_%28email%29
[ARF#T] http://tools.ietf.org/html/rfc5965#section-7.3
[RD] http://tools.ietf.org/html/draft-jdfalk-marf-reporting-discovery
[IANA-MARF]
http://www.iana.org/assignments/marf-parameters/marf-parameters.xhtml
[IANA-IODEF-PHISH]
http://www.iana.org/assignments/xml-registry/schema/iodef-phish-1.0.xsd
[PHISH] http://tools.ietf.org/html/rfc5901

---

From: <[email-masked]>
Date: Mon, 31 Jan 2011 09:15:06 -0500

Hello Alessandro,

Thank you for the additional background, that is very helpful.  I am tight on time today, so I will answer what I can quickly and then may follow up again later in the week.

IODEF reports can go to any entity that is handling/managing incidents and the response to those incidents.  I am guessing the use of Network Provider in the RID document is why it may have been interpreted to be for Network Providers only.  We included CSIRT as well, really meaning any computer security incident response teams.  This interpretation could extend to mail administrators.

I would like to see IODEF as an option in MARF and think the reasons and justification for this is growing.  Efforts like CYBEX from ITU.T are working to combine protocols to automate incident response.  Cloud is also driving the need for unified incident management between cloud providers.  With these efforts protocols that include SCAP, IODEF, and RID are getting tied together to automate incident response using a common format to represent incidents.  If you consider the CSIRT within an organization, having a single repository to discover, manage, report and track incidents provide several benefits.  There are efficiency gains when using a common reporting format in that the incident reports can be received and processed without the need to interpret multiple formats - possibly a different one for each type of incident.  In large organizations, the actions required from any one type of incident could be distributed throughout an organization.  If all of these incidents were managed in separate systems and different formats, it would become very difficult to normalize all incident types and identify trends across incident types.  A single attacker may launch multiple attacks that when correlated could assist in the forensics and also the actions taken to remediate since the combined attack types may have a greater impact and risk level to the organization.  If incidents are distributed internally using IODEF (to the right business units or administrators), the format could be leveraged to track the incident handling process.  The impact can also be associated with each attack type and trend analysis could also be performed to help determine areas of risk, need for additional funding, ROI benefits from new security measures implemented, etc. 

I actually had looked into MARF recently to see what was different, but have not combed through the definition to compare it to the IODEF + phraud schema extension yet.  If you have done this and see some areas that appear to be missing, please let me know as it may help to speed up my response.  Otherwise, I should be able to dig into the formats/incident representations later this week.

The reason we did not define a way to select the method of transport was that we have a defined transport definition (more can be defined) that uses HTTPS.  We were going to use BEEP, but that was changed to HTTPS in the last call process.  Additional transport mechanisms to transmit RID communications (RID is the wrapper for IODEF that provides security and types of communications that can occur) can be defined, nothing prevents the use of other transport protocols that get defined.

I think that just leaves one open question to see if IODEF and existing extensions accommodate the types of incidents handled by MARF.  If the phraud extension requires new fields, a document update could be possible.  If another extension needs to be defined for IODEF, that is possible as well.  There are 2 right now, one for financial transactions and one for phraud/phishing.  I have seen a note somewhere that a SPAM extension was needed, but will have to look through the schema to determine if that is the case.

Thanks for reaching out and bringing this to my attention.

Best regards,
Kathleen

-----Original Message-----
From: Alessandro Vesely [mailto:[email-masked]] 
Sent: Monday, January 31, 2011 8:43 AM
To: Moriarty, Kathleen
Cc: [email-masked]; [email-masked]; [email-masked]
Subject: Re: [marf] reporting-discovery

Hi Kathleen,

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Mon, 31 Jan 2011 10:19:15 -0800

There are a few things I can think of that would need to be done to support use of IODEF inside the ARF context.

The first would be to modify the reporting-discovery draft to include the possibility of advertising IODEF as a capability.  This seems a fairly easy change to make.

The second and somewhat more involved step would be to create another document that registers a new ARF report type which can accommodate an IODEF-formatted report.  However, at the moment ARF is predicated on the multipart/report media type (RFC3462) which stipulates that it must be the outermost media type used (preventing construction of a single message containing many reports) and further asserts the third MIME part in the report has to contain all or part of the body of the message of interest which appears to prevent the inclusion of an IODEF report.

It seems we may need a new media type overall to contain ARF reports might be useful.  I have an individual submission already created to remove the "must be outermost" constraint, specifically so that one could generate a single message that contains multiple ARF reports.  But maybe that should be abandoned in favour of making up a new media type that can accommodate ARF and IODEF more comfortably in the ways we want to use them.

---

From: Alessandro Vesely <[email-masked]>
Date: Mon, 31 Jan 2011 19:52:34 +0100

Hi Kathleen,
I appreciated your roundup of security efforts (although I didn't
quote it.)  Mail abuse is different inasmuch sending an email message
is not an intrusion/incident by itself.  I think the volume and
peculiarities of email abuse more than justify an ad hoc approach.
However, when email messages become evidence of full-blown incidents
they certainly deserve being managed with the appropriate tools, for
all the reasons that you described.

On 31/Jan/11 15:15, [email-masked] wrote:
> I actually had looked into MARF recently to see what was different,
> but have not combed through the definition to compare it to the
> IODEF + phraud schema extension yet.  If you have done this and see
> some areas that appear to be missing, please let me know as it may
> help to speed up my response.

ARF (RFC 5965), as you may have noticed, resembles a bounce, i.e. it
contains all or part of the original message as its final (third)
part.  This part may be readily included in the <phish:EmailMessage>
field of the phraud extension (RFC 5901).  The machine-readable
(second) part of ARF contains a few "name: value" lines, which can be
mapped to specific fields and/or included as <phish:EmailComments>,
unless new fields are added to the phraud extension for this very
purpose.  RFC 5965 chose not to specify the exact semantics of the
fields Reported-Domain and Reported-URI, so they depend on the
report's User-Agent and Version.  Also, some user-agents redact the
original messages, for privacy reasons.

> If the phraud extension requires new fields, a document update
> could be possible.

Yes, whenever we'll get to specifying that conversion.

> The reason we did not define a way to select the method of
> transport was that we have a defined transport definition (more can
> be defined) that uses HTTPS.

MARF uses SMTP as its elective transport.  This could be used to
quickly portray the difference between the respective approaches.

More than the transport, I'm puzzled by the discovery mechanisms.
Section 4 of RID (RFC 6045) starts with a nice informal description of
communication issues among providers.  Mailbox providers meet none of
those requirements, and I'd bet most of them don't think they should.
 This more heavily portraits the difference between MARF and IODEF,
and explains the need for discovery mechanisms.  I found mentions of
an X.cybex-disc paper, but have no idea of its content.
Do you think it may contain something useful to us?

> I have seen a note somewhere that a SPAM extension was needed, but
> will have to look through the schema to determine if that is the
> case.

Probably adding suitable type(s) would suffice.  The format defined by
the phraud extension seems to be adequate for reporting any kind of
abuse, irrespectively of its lure's malice.

---

From: Shmuel (Seymour J.) Metz <[email-masked]>
Date: Mon, 31 Jan 2011 14:29:57 -0500

In <[email-masked]>, on 01/31/2011
   at 02:42 PM, Alessandro Vesely <[email-masked]> said:

>suggests, is primarily used to report abuse,

It's more restrictive than that; the current RFC is for reporting
abuse to the sending network, not to networks providing, e.g., drop
boxes.

---

From: "J.D. Falk" <[email-masked]>
Date: Mon, 31 Jan 2011 15:08:19 -0800

> abuse to the sending network, not to networks providing, e.g., drop

There's no reason it couldn't be used to report drop boxes, though that's not a common practice today.  That's part of what the Reported-URI field was intended for.

Reported-URI: mailto:[email-masked]