Subject: [dmarc-ietf] [[email-masked]: [IAB] IETF 87 DMARC BOF evaluation]

From: Andrew Sullivan <[email-masked]>
Date: Tue, 30 Jul 2013 12:03:55 -0400

Dear colleagues,

Here's the report I sent to the IESG and IAB about the DMARC BoF.  I hope this
report seems correct to you.

Best.
A

----- Forwarded message from Andrew Sullivan <[email-masked]> -----

Date: Tue, 30 Jul 2013 11:00:54 -0400
From: Andrew Sullivan <[email-masked]>
To: [email-masked], [email-masked]
Subject: [IAB] IETF 87 DMARC BOF evaluation
List-Id: "Internet Architecture Board \(IAB\)" <iab.iab.org>

Dear colleagues,

The proposal for a DMARC WG is, in effect, to assume that a large
existing spec gets adopted on the standards track outside any WG, and
then to make a number of small changes to that existing spec in a WG.
The BoF started with quite a lot of background info about, in effect,
status of the existing work.  The basic spec seems to me pretty well
worked out (I felt rather like I was listening to presentations at the
end of a WG: small issues that needed to get nailed down).  But there
are little edge issues.

During the proposed charter discussion, I asked a number of questions
about why to set up a WG.  There's a cost to the IETF in having Yet
Another WG: we already have too much work.  To the extent that this
spec is in good shape and complete, why can't the group that developed
it complete the work?

If the problem is that there are these issues for which the consortium
that developed the specification doesn't have the expertise to solve,
then it's hard to understand why the base spec is out of scope in the
proposed charter.  

Since I heard lots of suggestions that the DMARC originators are
anxious to make the spec better, why not just put the DMARC spec on
the charter of the WG and do all the work?  I very strongly agree that
this basic work is an excellent basis for continued work.  But it
doesn't sound like it's done, and it seems to me therefore that the
existing running code is a 0.9 effort; the 1.0 effort needs to go
through the WG to complete.

In any case, I do see the potential for valuable work here, and it
does seem to me that the IETF is a natural place to do it since DMARC
builds on other enabling technologies created at the IETF.  At the
same time, I think the IESG ought to think very hard about its ability
to manage the number of WGs we already have, if the point is just to
fix things around the edges of the spec.  We have far too much work to
do already.  I saw significant appetite to work on this specification
in the room, though most of the people volunteering to write documents
were people who came from the existing DMARC work.

I am convinced, in any case, that the chartering effort is being a
little too cute about the relationship to the base spec, and I believe
that, if it is chartered on the basis of the existing proposed charter
before the base spec is published as an RFC, there will be major
procedural pain in someone's future.

Best,

A

-- 
Andrew Sullivan
[email-masked]

----- End forwarded message -----

-- 
Andrew Sullivan
[email-masked]

---

From: Trent Adams <[email-masked]>
Date: Wed, 31 Jul 2013 13:29:02 +0200

Andrew -

Thank you for your constructive and useful questions and comments during
the DMARC BOF in Berlin.  And thank you for taking the time to share what
you sent to the IAB and IESG.

To be clear, though, this evaluation is more of an opinion than a full
account of the BOF.  I wanted to be sure everyone on this list knows that
your note doesn't represent the consensus that emerged from the
participants in the BOF.  That's not to take away from your viewpoint, but
just to say that below I've inserted some commentary that I hope will help
provide a more complete picture (especially for those copied on this who
weren't privy to the conversations in the BOF).

To start, it should be clear that the consensus in the room (including the
remote participants in the Jabber Room) was in support of the proposed
Charter and associated work group items.  That's not to say support was
unanimous, as there were a couple of folks who indicated they preferred to
defer formation, but the support went far beyond rough consensus.

Hurray! And thanks to everyone who participated in the conversation.

Onward:

> end of a WG: small issues that needed to get nailed down).  But there

Something that you may have missed (though it was one of the major topics
of the Agenda) was the need to develop a "Using DMARC Guide" as presented
by Dave Crocker.  The presentation made clear that there are a lot of
interesting areas for exploration of technical choices to be made when
implementing the base DMARC specification.

I won't reiterate the items identified and discussed at length (they're
covered within the proposed draft as well as summarized in the presentation
slides).  Rather, I wanted to ensure that you understood that we've been
advised by a number of long-time IETF folks that the specification itself,
to be useful as a long-lived Standards Track document, should not spend
time discussing technical operational configuration options.  Rather, they
should exist in a separate document (or documents).

The "Using DMARC Guide" is the first foray into that space, and it is
packed with technical discussions that warrant much work.  Given what you
said here, it doesn't seem like it was clear that this is a primary driver
for the proposed DMARC WG.  Yes, I spoke about some interesting extensions
that could be added to the utility of the base specification, but they were
one component of the overall work for the proposed DMARC WG, but not the
entire (nor primary) focus.

BTW - You know this since you were in the room, but for the people who
couldn't make it, there was a fantastic discussion about
internationalization that was brought up on the floor as a possible work
item.  In fact, there were a few "+1" comments in support (with a strong
suggestion that this would be worth dealing with in the "Using DMARC
Guide").

> spec is in good shape and complete, why can't the group that developed

It is great to see you share that you asked this question, but it's too bad
you didn't also include the answer.  In the room, we replied to this
excellent question (which we've been asked a number of times during this
journey) that the original authors have come to a point in our
deliberations where we're seeking counsel from the broader community.

Since at least 2011 we've been advised that we should be openly engaging
with the broader community.  At that time we weren't quite ready, though we
were actively heading in that direction, we just needed to tie off some
loose ends.  At the very earliest opportunity, we began to steer the ship
into (what we hoped would be) the welcoming waters of the IETF.

At the advice of many (including public calls by an Area Director), we shut
down private conversations a few months ago about the technical development
of the specification and moved the discussion to the public IETF list.
 This move means that we are essentially taking the expert advice and can't
really look back now.

The path is forward and open, not back and closed.

> then it's hard to understand why the base spec is out of scope in the

The base DMARC specification is only closed insofar as expert reviewers
convince Barry Leiba, as our possible AD Sponsor, that it is ready to be
moved to Standards Track (extended) last call.  We've heard from a number
of reviewers on the list already that what remains is largely editorial (or
else items that can be disposed of during the last call phase).

The reason we didn't include the base specification in the proposed DMARC
WG Charter is that we have been advised that it is distinctly possible that
there's not enough technical work to warrant being included.

We did, however, anticipate that it may be necessary to explore work on the
base specification at some time in the future.    If that happens during
the lifetime of the DMARC WG, the proposed Charter calls out the
possibility of re-chartering in order to tackle it.

> through the WG to complete.

If you haven't done so already, this seems like a fantastic opportunity for
us to ask you to provide a detailed review of the specification in response
to Barry's call for expert input.  I'm sure your detailed analysis would be
greatly appreciated as he weighs the options and decides whether he feels
comfortable sponsoring the specification.

> builds on other enabling technologies created at the IETF.

Indeed.  It's gratifying to see the work all come together under a single
umbrella rather than as scattered work.

> At the

On the one hand you say, "we have far too much work to do already" and on
the other hand you say "most of the people volunteering . . . came from the
existing DMARC work."  Couple that with the fact that we're told that we
should have brought this work in earlier (to a system that apparently has
little energy for more work).

>From my read, that means that the IETF gets a net increase in capacity with
the influx of new, energetic contributors to accomplish real, meaningful
work.  Further, given that there is already wide adoption of the pre-IETF
specification, it also means that the IETF didn't have to expend it's
paucity of resources getting us to this point.  Win!

In the end, the merit will win out.  Either the proposed DMARC WG Charter
is accepted given the technical work, knowing that there is
ready-and-willing volunteers committed to the work, or not.  I trust the
IETF process enough to know that other issues won't cloud the judgement.

For your consideration,
Trent

> _______________________________________________
> dmarc mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/dmarc
>

-- 
J. Trent Adams
=jtrentadams

Profile: http://www.mediaslate.org/jtrentadams/
LinkedIN: http://www.linkedin.com/in/jtrentadams
Twitter: http://twitter.com/jtrentadams

---

From: Andrew Sullivan <[email-masked]>
Date: Wed, 31 Jul 2013 07:44:03 -0400

On Wed, Jul 31, 2013 at 01:29:02PM +0200, Trent Adams wrote:
> account of the BOF.

Totally correct.  The note I sent is a report from an IAB member
"covering" a BoF.  The point of this is for the IAB member to raise
questions, issues, views, and so on _from that IAB member's point of
view_ for the IESG to consider.

The BoF report is prepared by the Chair(s) and is supposed to discuss
what consensus emerged and so on.  And yes, it's just one view.  The
IESG knows that, and is as likely as not to discount any view that
comes from me, if they're sane. :-)

In any case, yes, my remarks are _not_ a complete view of what the
exchanges were.  They're really intended to raise issues/concerns/&c.
I try to keep them as short as possible because the IESG has to read
all of these, sometimes from multiple IAB members, for each BoF.  I do
sometimes sacrifice completeness to brevity and timliness in that
case, and I do not wish anyone to think that I wasn't listening to the
important details you've correctly filled in.  

And you're quite right that an influx of people who come and want to
do work here -- particularly operators -- makes me quite happy.

Best,

A

-- 
Andrew Sullivan
[email-masked]

---

From: Dave Crocker <[email-masked]>
Date: Wed, 31 Jul 2013 14:24:22 +0200

My own .02e:

Because we are trying to navigate things in an unusual way, we have an 
especially challenging obligation to be precise in what we ask for, how 
we ask for it, and how we explain things.  In spite of our efforts to do 
all that, I think some confusion persists.

Yours and Eliot's comments are entirely reasonable, IMO.  As you'll see, 
I disagree with an essential point about the charter, but it's exactly 
the one that would be expected, both as a point to be raised (by you 
guys) and as a disagreement from anyone in favor of what's been proposed.

So...

{ caveat:  I think my 'reply' mechanism or my reading has confused 
Andrew text with Trent text.  So I'm responding to the text segments, 
without worrying who typed them. }

> On Tue, Jul 30, 2013 at 6:03 PM, Andrew Sullivan <[email-masked]
> <mailto:[email-masked]>> wrote:

> packed with technical discussions that warrant much work.  Given what
> you said here, it doesn't seem like it was clear that this is a primary
> driver for the proposed DMARC WG.  Yes, I spoke about some interesting
> extensions that could be added to the utility of the base specification,
> but they were one component of the overall work for the proposed DMARC
> WG, but not the entire (nor primary) focus.

That matches my own view of things...

> suggestion that this would be worth dealing with in the "Using DMARC

The problem with that process was that it had no actual substance.  The 
topic is always an important one and it is often underserved, if served 
at all.  But its application to the DMARC spec was entirely ad hoc, 
partly in the way concerns were raised or asserted, and partly in how 
they were processed.

I see two key points, concerning the base spec:

      1)  The document does have an 'internationalization' bit of text; 
whether it's enough I can't answer.

      2) We need to have qualified folk do an audit of the spec for 
this; if they come up with /specific/ required changes, then of course 
we have to do them.

Only after #2 has occurred and produce a targeted requirement will there 
be a basis for saying that the base spec needs internationalization work.

As for the draft BCP and adding internationalization discussion -- well, 
heck yeah.

> It is great to see you share that you asked this question, but it's too
> bad you didn't also include the answer.

The problem with the question is that it was asked in an ad hoc manner, 
but is instead a strategic, global topic for the entire IETF organization.

We do need to deal with it but we have no meaningful discussion base in 
the IETF -- and have in fact repeatedly resisted constructive discussion 
about it for perhaps 20 years -- and no metrics or even any shared frame 
of reference.

This reduces the question to an ad hoc challenge to "say random stuff 
that sounds compelling", which might make for good theater but isn't a 
serious evaluation process.

The simple fact is that dmarc is a topic relevant to the scope of IETF 
work, with people who want to pursue it and some basis for believing 
useful work can be done.  If we have a more stringent evaluation metric 
for new working groups, please point us at it.

> moved to Standards Track (extended) last call.

This is an absolutely key point (and nicely phrased, I might add.)

> The reason we didn't include the base specification in the proposed
> DMARC WG Charter is that we have been advised that it is distinctly
> possible that there's not enough technical work to warrant being included.

This needs to be stated more affirmatively:  After multiple queries on 
two public lists, there have been no requests for technical changes to 
the base specification that have developed any traction.

If and when that changes, the path for the base spec will (of course) be 
re-evaluated.

> We did, however, anticipate that it may be necessary to explore work on
> the base specification at some time in the future.    If that happens
> during the lifetime of the DMARC WG, the proposed Charter calls out the

+1

>     through the WG to complete.

If it doesn't sound like it's done, then we not have phrased things 
properly.  To date, we have no evidence that the base spec needs 
technical changes.  We now have John Levine asserting that some such 
work does need doing; if it's evaluated as indeed needing it, it will 
indeed be added to the charter (assuming the fixes can't be done instantly.)

Absent that, it does not make sense charter a wg to work on a spec when 
there is no technical work known to be needed on it.

One point of confusion concerns the various discussions of /possible/ 
extensions, including those cited in the draft charter.  For one thing, 
we don't know whether they will be productive.  For another, we don't 
expect them to affect the base.  If it is evaluated that any of them 
/do/ affect the base, then the work will indeed be added to the wg.

Absent such specifics, adding it to the wg charter is only an invitation 
to de-stabilize the spec.

In separate conversations with others, I've noted an occasional lack of 
sensitivities to market perceptions of specification stability.  The 
reality is that the market sees a spec going into a working group as a 
very strong argument against using the spec now.  After all, it's going 
to change and we don't know when it will stop changing.  This can have a 
very serious effect on market adoption momentum.

> If you haven't done so already, this seems like a fantastic opportunity
> for us to ask you to provide a detailed review of the specification in
> response to Barry's call for expert input.

+1

>     fix things around the edges of the spec.

Sure it should.  In a careful, principled and equitable fashion...

d/

-- 
Dave Crocker
Brandenburg InternetWorking
bbiw.net