Subject: [mpowr] Summary of Discussion on Reforming IETF Quality Control Process

From: "James Kempf" <[email-masked]>
Date: Fri, 9 Jan 2004 16:40:01 -0800

I've written a draft summarizing the discussions on reforming the IETF
quality control process that started on mpowr in December and wandered to
solutions in January. It's here:

http://www.geocities.com/kempf42/draft-list-quality-control-00.txt

I am not exactly sure what to do with the draft at this point, but would be
interested in hearing feedback. There has been a preference expressed for
discussing the issue on solutions rather than on either mpowr or icar, but
I'm on all three lists and I'll try to respond as time allows. I'd also be
interested in hearing suggestions about how to proceed. If this doesn't look
like a valuable activity, we should apply the procedure described in the
draft and terminate it early. :-)

            jak

_______________________________________________
mpowr mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/mpowr

---

From: Margaret Wasserman <[email-masked]>
Date: Fri, 09 Jan 2004 20:41:09 -0500

Hi James,

I just read your proposal.  Thanks for posting it.

Although I may have more detailed questions later, I have a
couple of quick clarification questions now...

What are the criteria for deciding whether a particular document
requires full IESG review?  And, when would this be decided -- at
charter time, when the document is accepted as a WG item, during
WG Last Call, when the shepherding AD reviews the status of the
document, or at some other time?

In dispute resolution, you seem to indicate that the shepherding
AD will take all disputes to the full IESG.  Currently, the
shepherding AD (or two ADs in an area) make an attempt to resolve
any dispute before it is brought to the full IESG.  Is it your
intention to change that?

Do you have any sort of transition plan for how we could move
to this approach?  Are there useful experiments that we could
run to see if it would work?

Margaret

At 04:40 PM 1/9/2004 -0800, James Kempf wrote:
>Solutions mailing list
>[email-masked]
>http://eikenes.alvestrand.no/mailman/listinfo/solutions

---

From: "James Kempf" <[email-masked]>
Date: Fri, 9 Jan 2004 21:05:24 -0800

Hi Margaret,

That was quick! Here are a couple of quick answers, but if you think this
draft might have legs, I'll get an issues page up on it.

> requires full IESG review?

It is up to the IESG to decide that. The WG can ask for a full review if
they think it might be necessary, but the IESG makes the decision. Also, the
shepherding AD can ask for a full review if s(he) thinks its necessary. In
general, I think the IESG would want to review drafts from WGs that are
proposing new technology or major changes in old technology, but I did not
think it would be such a good idea to be that explicit because it would
decrease the attractiveness of serving on the review board, and, as a
practical matter, the IESG may want at some point to have the review board
do a review on some draft that proposes new technology if it's a small
change or a major change in old technology if it isn't all the different or
if the WG is composed of people who are known to be reliable and world class
experts in the field.

>And, when would this be decided -- at

In the normal case, as soon as the draft showed up as a work item on the
WG's goals list, which is what the process draft should say because some WG
drafts are added after the charter has been written. A review would have to
be done before the draft became a WG draft in any case. The idea here is to
inject a note of predictability into the review process, so the chair, WG,
and shepherding AD know exactly what to do when the time comes for the
review.

However, in exceptional cases, typically when the shepherding AD has
detected that the WG is off track or has reason to believe that the
solutions being discussed by the WG may be impractical or otherwise flawed,
the shepherding AD can call for a review and include an IESG review of
documents, if talking with the WG fails to reorient the WG.

> intention to change that?

Yes, I believe that is probably a better idea and I will add that to the
document. No reason for full IESG intervention if the WG and reviewers can
come to an accommodation with the AD's help.

> run to see if it would work?

One way to start would be to pick a few WGs and give it a try. The list of
SIRS are obvious candidates for initial Reviewers. Obviously, if the process
is applied to some WGs that exist initially, they will not be able to set up
the review process when they are initially formed. I'd say picking groups
such as DHC where many of the drafts involve proposals for new options or
other small changes would be one way to test the review board process, I'm
not sure which ones would be appropriate for the new/deeply changed
technology case. The experiment could be run for, say, a year and closely
monitored to see how much work it really did eliminate for the IESG and
whether it really did free up more time for ADs to spend working with their
WGs, then tuned as appropriate or even discarded if it isn't working.

            jak

---

From: Alex Rousskov <[email-masked]>
Date: Fri, 9 Jan 2004 22:43:57 -0700 (MST)

On Fri, 9 Jan 2004, James Kempf wrote:

> It is up to the IESG to decide that.

I want to clarify this because Margeret's question might be based on
the wrong assumption, possibly causing misinterpretation of the
correct answer.

There should be no "requires full IESG review" flag or state for any
document. IESG is not a special case when it comes to review. IESG or
any single AD can submit a review for any document that is up for
review, at any time. This is no different from any IETF participant
submitting a review. If IESG feels that a particular document needs
full IESG review, it is their internal business, invisible and
unpredictable to others (in general), until they submit a review. In
general, one does not know a priori whether a document will be
reviewed by IESG until the IESG submits the review.

Now, if there is a conflict that WG and a reviewer cannot resolve
despite all the negotiation efforts, then the document automatically
goes to IESG for the final conflict resolution. Such resolution may
require full IESG review, but it is internal IESG business how to
approach that. The final IESG decision is documented, of course. Note
that "a reviewer" above may be IESG (but it is not a special case).

Thus, IESG has special formal powers when it comes to conflict
resolution, but does not have (and does not need!) any special powers
or exceptions when it comes to document review. This scheme is both
simple and gives IESG full flexibility when it comes to selecting
"full IESG review" targets. We do not need to spend time documenting
what those targets might be.

Alex.

---

From: "Joel M. Halpern" <[email-masked]>
Date: Sat, 10 Jan 2004 01:42:43 -0500

I may be missing something, but the problem I see reading this is that this 
gives the IESG no assistance in deciding which documents it needs to 
review.  There was another proposal which I read to say ~the IESG needs to 
review documents for which there is disagreement about the results of other 
reviews.~  While one can argue about whether that is good or not, it at 
least spells out when the IESG needs to review a document, and when it does 
not need to.
Obviously, if we can lighten the load enough that the IESG has time to 
review more than it needs to, that is great and they should do so.
Equally, some IESG members will review some documents over and above such a 
"requirement", since they will be interested.
But my understanding of one aspect of this discussion is to help replace 
the current ~the IESG must review everything~ with a useful alternative 
review procedure, and a useful definition of which things the IESG needs to 
review.
(Among other things, such a reduction should help enable the necessary IESG 
reviews to occur in a more timely fashion.)
The text below seems to say taht there are no documents the IESG needs to 
review.  While it would be nice to arrive in such a state, I doubt we can 
get there in one jump.

Yours,
Joel M. Halpern

PS: Can we pick one of the three lists for this?