Subject: [mpowr] Why MPOWR?

From: Bernard Aboba <[email-masked]>
Date: Wed, 4 Feb 2004 08:54:05 -0800 (PST)

If MPOWR is a solution, I would like to understand what the problem is.

To me, the issue is not IESG overload.  It is the inability of the IETF to
deliver timely and relevant protocol standards.  I can see how the
standards track reforms and perhaps even cross area review might address
that problem (though I'm skeptical about whether either will make a major
difference), but MPOWR seems largely tangential.

In general, I'd prefer to see the reform effort make an effort to pick a
few success metrics and make substantive progress on them.  The IETF is
beginning to show signs of a vicious cycle, which when started, can become
very hard to break.  You don't break a cycle like that and start on an
upward path again without a very clear idea of what you're trying to do,
and what reforms can make a meaningful difference in the key success
metrics, enough to make people stand up and take notice.

A month ago, I attended the IEEE 802 meeting in Vancouver, and it
resembled the IETF meetings of several years ago.  Not just in spirit --
there were lots of former IETF attendees there, some former
chairs of WGs, and even ex-IAB and IESG members.  Many of them now
consider IEEE 802 their primary standards body, not IETF.

If we're going to bring those people back, we are going to need to do
something to make the IETF a place where people feel convinced that they
can get important work done in a timely way.

Some simple reforms could probably make a measurable difference in the
total time from a -00 to an RFC in the short term.  For example, one could
cut considerable time off the end of the process by addressing the
issues with IANA and also by having a seminar at each IETF about reference
dependencies.  In examining the RFC Editor Queue, it is fairly clear to me
that many authors do not understand what a normative dependency is, and as
a result their drafts languish in the queue awaiting resolution of
dependencies which are not really normative.

There are even some drafts in the queue with no prospects for
publication, since they have normative dependencies on drafts that were
abandoned long ago and will never be completed (one of the MIDCOM
documents is an example of this).  A simple education program on
normative references could probably cut several months off the average
completion time.

I also believe we could cut significant time off the beginning of the
process by streamlining the BOF process and moving more quickly to form
"study groups".  There are quite a few cases where it has taken more than
a year to form a working group, and even at least one case where formation
took several years.  Streamlining the BOF process seems quite likely to
shave several months without a lot of effort.

Making issue tracking facilities generally available to WGs is also
something that is likely to cut significant time off the process.  In at
least one WG I've been involved in, it can be argued that the WG process
only began to become effective *after* IESG review, when a tracking system
was installed.  More real work was done in 12 months with an issue
tracking system than was accomplished in the previous two years.  So
significant productivity increases seem likely.

A few months here, a few months there... soon you're talking about
significant time savings.  The difference between a standard that takes
3.5 years and one that takes 5 years is in practice, large.  The former
is late, but perhaps tolerable if the quality is high.  The latter is
likely to have arrived after the market has already chosen a (proprietary)
alternative.

_______________________________________________
mpowr mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/mpowr

---

From: Melinda Shore <[email-masked]>
Date: Sat, 7 Feb 2004 09:13:34 -0500

On Wednesday, February 4, 2004, at 11:54 AM, Bernard Aboba wrote:
> documents is an example of this).

Note that the references in the midcom draft in question have been
corrected and the revised document is in the RFC editor's hands.  The
problem in this particular case is that because we don't have a way
of flagging documents as being dependencies there's no way to know
who to notify when work on a document is stopped or the document is
dropped completely.  This isn't an mpowr problem but rather a
tools problem.  It would be useful to tag unpublished documents that
are cited in somebody else's normative references so that when the
documents change status notifications can be propagated.  We're nowhere
near being able to put that kind of capability in the hands of
document editors, which is where it seems to me to belong.

Melinda

---

From: John C Klensin <[email-masked]>
Date: Sat, 07 Feb 2004 11:26:40 -0500

I want to seize on, and expand a bit on, one aspect of Bernard's 
note (I generally agree with all of it)...

--On Wednesday, 04 February, 2004 08:54 -0800 Bernard Aboba 
<[email-masked]> wrote:

>...
> I also believe we could cut significant time off the beginning
> of the process by streamlining the BOF process and moving more
> quickly to form "study groups".  There are quite a few cases
> where it has taken more than a year to form a working group,
> and even at least one case where formation took several years.
> Streamlining the BOF process seems quite likely to shave
> several months without a lot of effort.
>...

This, unfortunately, is another example of the process-rot 
problem that I've been trying to point out.  A decade ago, a BOF 
(much less a couple of BOFs) were not requirements for creating 
a WG.   BOFs were used if there was a need to get more 
information about interest and focus; if that need did not 
appear to exist, we went directly into a charter process.   If a 
BOF is unlikely to produce new information or insights, moving 
directly to chartering at least avoids having the nit-picking 
and grand philosophy arguments twice (or more) --once as to 
whether the BOF(s) should be permitted and with what agenda(s) 
and again over the details of the charter.  More about the grand 
philosophy arguments below.

In addition, there was a somewhat more relaxed attitude toward 
chartering WGs.  Several ADs took the position that, if a group 
seemed coherent but was unfocused, it was rational to charter 
them as a WG and watch them closely for a while to determine 
whether they were going to produce something useful or go off 
into the weeds.  If the latter, they got pruned, if the former, 
they flowered.  The assumption was that the best way to evaluate 
whether a WG was going to do something useful and competent was 
to let it try.  This direct approach was believed to be much 
more efficient, and less likely to waste a lot of everyone's 
time, than interminable discussions about charters, wondering 
whether WG proponents were trying to tune charters to what 
people wanted to hear even while planning something else, 
speculation about results, and so on.

Growth, and the "just too many WGs" problem would make that 
model more difficult if applied today.  More important, the IESG 
became shy of using it, probably for several reasons.  Certainly 
one of them was hearing from too many WGs "we struggled to get a 
charter, you chartered us, we did all this work, so we are 
_entitled_ to complete our work and have our output 
standardized".  Of course, the more time and effort goes into 
the chartering process, the stronger that argument gets.   I 
also assume that most ADs would prefer to spend time managing/ 
coordinating WGs that are trying to do technical work --and 
shutting down those that are hopelessly wedged-- than to spend 
the same time in charter debates, especially philosophical ones. 
I might even suggest that people who don't have that preference 
are unsuitable to serve as ADs.

It is important to note that the key procedures have not changed 
at all in the last decade.  The evolution to "you need a BOF or 
two" and "charters are subject to long community debate" both 
occurred without formal process changes.  If we don't like them 
--and they certainly add a lot to the time between "proposal to 
do work" and "finished product"-- they can presumably be undone 
by the same quiet and efficient mechanisms.

Now these changes occurred mostly because of IESG perceptions (I 
think accurate ones) that the community wanted more input into 
early-stage WG establishment efforts.   Well, that greater 
opportunity for discussion and greater openness now has a track 
record.  The track record is dismal, at least with regard to a 
"how long does it take to get things done" criterion.  We start 
a discussion.  It generates a few comments and, especially in 
"hot button" areas, a lot of noise (see below).  So a BOF is 
held, either because everyone believes by now that they are a 
requirement or to see if a face to face meeting resolves some of 
the controversy.   It doesn't resolve the controversy, partially 
because it is easier for someone to block progress by seizing a 
microphone and grandstanding than it is to do so on a mailing 
list -- if only because most of us have adopted "delete before 
reading" approaches to comments beyond a certain number by some 
people on some threads (that has its own bad effects, but they 
are another matter).  The mailing list is then filled, for the 
next few months, by the same noise, the same passionate 
arguments repeated many times, and by debates about what the 
first BOF actually concluded.  So a second BOF is held in the 
vain hope of sorting that out, and the process repeats.   If the 
effort then starts a charter process, the same arguments are 
replayed again, typically with little evidence that the BOFs 
were held or accomplished anything.  But six months to a year go 
down the rathole -- much time spent, little real work done.

If we want this to stop, we need to make it _very_ clear to the 
IESG, clear enough to overwhelm the noise, that we are tired of 
it.  No more BOFs, and especially no second BOFs, unless it is 
clear that useful information is likely to come out of them. 
An accelerated chartering process with clear community support 
for shutting down WGs that looked marginal at charter time, were 
given a chance anyway, but aren't producing (there may be 
elements of the O'Dell-Klensin and/or Huston-Rose proposals from 
early in this reform process that might be useful here).

And we need to be able to try that way of doing things (again?) 
without getting bogged down for months in BOFs, chartering 
discussions, and philosophical arguments about how to plan the 
process to do so.  We didn't need any of that process to get 
into this mess, and we shouldn't have to exercise the symptoms 
of the mess to get us out of it.   (A couple of us are finishing 
up an I-D that suggests a radical way out of this impasse.  It 
will be queued before the Monday deadline.   But it doesn't 
suggest anything that couldn't be done tomorrow if the IESG sees 
a requirement and mandate for it, sees community rough consensus 
about that mandate, and is actually inclined to make changes 
rather than fostering debate about them.)

Finally, that comment about philosophical debates.  There are a 
number of hard philosophical and/or architectural questions 
facing the IETF and the Internet.  They include a number of 
issues surrounding the NAT and middlebox questions; how much we 
are willing to distort the operations of well-established 
protocols in order to provide small, possibly-useful, patches 
for high-impact problems that arguably really lie elsewhere 
(spam fighting and the impact of email-spread and 
http/html-embedded malware being good examples here); questions 
about configuration alternatives; variations on the "open 
source" and/or "free software" debates; and so on.  They are 
important issues.  We need, IMO, to face and debate them in 
clear and open ways and, ideally, reach some conclusions.  But 
passions about them run high on both (all?) sides.  If every 
attempt to charter a WG or review a proposal for standardization 
that might infringe on one of those "hot button" areas restarts 
the debate in the context of that charter or proposal, things 
will move slowly --perhaps to the point that we will never get 
anything done-- and the passion of those debates will drown out 
any real analysis and discussion of the specifics of the 
proposal.   Unfortunately, we have ample worked examples of that 
pattern, more than enough that we should be learning from them. 
The people who are doing this are presumably sufficiently 
self-aware to know who they are.  In the interest of progress in 
the IETF --of having an IETF that, a few years from now, will 
[still be?] perceived as able to do useful work -- they need to 
stop attacking individual WG proposals and work on a more 
general debate and discussion.  And, if they can't or won't 
--especially after the general decisions of what the IETF will 
and will not do in a particular area have been made-- then the 
community needs to understand that this behavior, no matter how 
well motivated, is as destructive to progress in the IETF as 
personal attacks, repeated off-topic postings, and so on, and 
deal with them accordingly.

     john

p.s. While triggered by a note on MPOWR, this note probably is 
applicable across much of the "solutions" spectrum.  If someone 
feels that it has enough value to call attention to it on some 
other list, feel free.

---

From: "Joel M. Halpern" <[email-masked]>
Date: Sat, 07 Feb 2004 18:32:20 -0500

The problem I see with this is that the IESG (and IAB, and nomcom) are 
caught in the middle of a distinctly bimodal community.  A significant 
portion of the community (probably myself included) would be very happy 
with an approach that said "charter more easily, clsoe if the results / 
work starts to look bad."  The properties of this could be paraphrase as a 
direction to the leadership of "use you judgement." with the balance being 
that if they exhibit bad judgement the nomcom replaces them.
Another (apparently equally large, and clearly equally vociferous) portion 
of the community is extremely uncomfortable with the leadership just using 
their judgement.  Any time such judgement is used, they request detailed 
explanations.  If they disagree, they point to different behavior in the 
past an demand consistency.  This point of view is as defensible as the 
first one.

However, I can not imagine how leadership should behave when there are 
equally strong demands for two such antithetical positions.

Yours,
Joel M. Halpern

PS: Yes, I think this aspect at least belongs on MPOWR, because this drives 
very directly the question of allowing WG Chairs to use their judgement.

At 11:26 AM 2/7/2004 -0500, John C Klensin wrote:
>If we want this to stop, we need to make it _very_ clear to the IESG, 
>clear enough to overwhelm the noise, that we are tired of it.  No more 
>BOFs, and especially no second BOFs, unless it is clear that useful 
>information is likely to come out of them. An accelerated chartering 
>process with clear community support for shutting down WGs that looked 
>marginal at charter time, were given a chance anyway, but aren't producing 
>(there may be elements of the O'Dell-Klensin and/or Huston-Rose proposals 
>from early in this reform process that might be useful here).

---

From: Pekka Savola <[email-masked]>
Date: Sun, 8 Feb 2004 09:01:48 +0200 (EET)

Hi,

I agree with Joel's comments, so I don't repeat them here, but only 
want to spell out an issue explicitly..

> early in this reform process that might be useful here).

.. you seem to state "aren't producing" as a criterion for shutting 
down a WG.  I'm not sure if this is intended, but the problem seems to 
be more generic than that.

For example,

 a) the WG could get nothing done, because of the lack of 
interested/committed people, or do so only slowly;

 b) the WG might be getting something done, but the stuff produced 
would not seem to be a useful approach (e.g., architecturally wrong 
choice after the fast-track WG process began)

 c) the WG might be producing something, but the results will be 
irrelevant, as it lacks the support/commitment from major vendors, 
which would be a requirement for getting any useful work done (e.g., 
forces WG could be one example).

There are probably more issues to spell out, but "aren't producing" 
could be interpreted in a narrow fashion ("aren't producing 
anything"), or broader ("aren't producing good, useful output").

It is important to see the distinction, and the unavoidable pitfalls 
of the latter interpretation, as b) and c) are likely to be another 
(subjective) judgment call.

Thus it seems to make sense to "charter carefully because you've got
to live with the results", and "charter with sufficient number of
checkpoints to check whether the WG is producing the right stuff".  
However, this doesn't necessarily mean the chartering process needs to
be slow..

-- 
Pekka Savola                 "You each name yourselves king, yet the
Netcore Oy                    kingdom bleeds."
Systems. Networks. Security. -- George R.R. Martin: A Clash of Kings

---

From: "Spencer Dawkins" <[email-masked]>
Date: Sun, 8 Feb 2004 07:10:40 -0600

Hi, Pekka,

If I understand the conditions you named so far might be summarized as

- not doing much of anything,

- doing the wrong thing,

- not doing anything that matters.

Taking on the third condition seems impossible as long as we remain
commited to an IETF with individual participation and no formal
membership. If 30 people from five major vendors participate actively
in mailing list discussions and produce documents in a timely fashion,
this says nothing about the willingness of those vendors to produce
products based on those documents or the willingness of their
customers to deploy these products. IPv6.

This is a much easier call in other standards bodies. The assumption
is that if you have people working in 3GPP, where companies pay tens
of thousands of dollars to participate, and meeting participation is
required to get anything done (every couple of months, for a week at
the time, scheduled around the world), you really could take a nose
count.

3GPP SA2, which interacts heavily with IETF, has six week-long
meetings scheduled this year (2 in France, plus China, South Korea,
Canada, and the US), and its recommendations are approved at four more
weeklong SA plenaries (two in the US, plus South Korea and Greece).

It is a lot easier for people to participate in the IETF with minimal
backing from employers. We seem to like it that way, for various
reasons, and we may be doing it the right way.

But this does mean we can't use "lacks the support/commitment from
major vendors" as a reason to kill something.

Spencer

From: "Pekka Savola" <[email-masked]>
To: "John C Klensin" <[email-masked]>
Cc: <[email-masked]>
Sent: Sunday, February 08, 2004 1:01 AM
Subject: Re: [mpowr] Why MPOWR?

---

From: John C Klensin <[email-masked]>
Date: Wed, 11 Feb 2004 13:51:33 -0500

Joel, and Pekka,

As you have probably guessed, this discussion was one of the 
inspirations for the "trust the IESG" 
(draft-klensin-problem-july15-00.txt) and "cannot trust the 
IESG" (draft-klensin-problem-planB-00.txt) drafts.  We need to 
get unstuck from the dichotomy of beliefs that Joel identifies, 
or, to be a little melodramatic, we are dead in the water and 
nothing else will make any difference.

While I'm appalled at some of the procedural changes the IESG 
has made, apparently without consulting anyone or making 
announcements (see my comments some months ago (on the 
Problem-Statement list, I think) about playing "gotcha" with the 
community), my personal view is that "trust the IESG" (aka "use 
your judgment") is the only plausible answer because:

	* No amount of charter-bashing is ever, in progress,
	going to define and eliminate all of the ways in which a
	WG might go wrong.  So someone --and, in our current
	model, it has to be the IESG or individual ADs-- has got
	to, in the last analysis, have the ability to make a
	judgment about overall community consensus and the
	health of the IETF and the Internet and shut off WGs who
	are not productive (see below).  And, for me at least,
	one clear corollary to that is that dragging out the
	chartering process is, beyond some point, a poor use of
	time.  When that point is reached is also, inevitably, a
	judgment call.  I think the community has driven the
	IESG to be too liberal about letting those discussions
	continue endlessly (ok, some IESG members have
	"helped"), and that we need to apply some corrections in
	that area.
	
	* I've "been there and done that" -- on the IESG, as a
	somewhat internal observer of the IESG from the IAB, and
	in management and overview positions in a number of
	other standards bodies.  My conclusion from those
	experiences is that any scheme that relies on in-depth
	community input and decision-making on every issue,
	rather than on the IESG's ability to interpret consensus
	and to lead, is going to either not work or will require
	us to adopt a formal system or membership and voting.
	
	* Given how much fun most of the people who are doing
	significant work in the IETF seem to find fussing with
	process and administrative issues, I'd much rather find
	13 or so hard-working and dedicated people who are
	willing to deal with this stuff and waste their time (on
	a "someone has to do it" basis) than waste the time of
	the thousands of the rest of us.  And, if an IESG member
	starts liking the process stuff too much, I hope the
	Nomcom is able to notice that and rotate him or her out
	-- in the interest of preserving the sanity of a valued
	colleague, if nothing else.

Sometimes, if we want an IESG-led model, we are going to have to 
shrug some things off as falling into the "sometimes consistency 
is the refuge of small minds" category.  Excessive procedural 
rigidity ultimately benefits neither the IESG, nor the IETF, nor 
the Internet.

Re "not producing", I intended that in the broadest way 
possible, to include everything Pekka lists and also "is taking 
up, and is likely to continue to take up, more energy to manage 
and keep pointed in the right direction than their outputs are 
likely to be worth ... either on an absolute scale or in 
comparison to other Area or IETF activities".   If we don't have 
a mechanism for shutting down a WG because it is no longer clear 
that the value of its likely product is worth the costs in 
management, oversight, reviews, meeting slots, etc., then, 
again, things are probably hopeless because every charter must 
be perfect in both definition and foresight.

     john

--On Saturday, 07 February, 2004 18:32 -0500 "Joel M. Halpern" 
<[email-masked]> wrote:

> The problem I see with this is that the IESG (and IAB, and
> nomcom) are caught in the middle of a distinctly bimodal
> community.  A significant portion of the community (probably
> myself included) would be very happy with an approach that
> said "charter more easily, clsoe if the results / work starts
> to look bad."  The properties of this could be paraphrase as a
> direction to the leadership of "use you judgement." with the
> balance being that if they exhibit bad judgement the nomcom
> replaces them.
> Another (apparently equally large, and clearly equally
> vociferous) portion of the community is extremely
> uncomfortable with the leadership just using their judgement.
> Any time such judgement is used, they request detailed
> explanations.  If they disagree, they point to different
> behavior in the past an demand consistency.  This point of
> view is as defensible as the first one.
>...

--On Sunday, 08 February, 2004 09:01 +0200 Pekka Savola 
<[email-masked]> wrote:

>
> .. you seem to state "aren't producing" as a criterion for
> shutting  down a WG.  I'm not sure if this is intended, but
> the problem seems to  be more generic than that.
>
> For example,
>
>  b) the WG might be getting something done, but the stuff
> produced  would not seem to be a useful approach (e.g.,
> architecturally wrong  choice after the fast-track WG process
> began)
>
>  c) the WG might be producing something, but the results will
> be  irrelevant, as it lacks the support/commitment from major
> vendors,  which would be a requirement for getting any useful
> work done (e.g.,  forces WG could be one example).
>
> There are probably more issues to spell out, but "aren't
> producing"  could be interpreted in a narrow fashion ("aren't
> producing  anything"), or broader ("aren't producing good,
> useful output").
>
> It is important to see the distinction, and the unavoidable
> pitfalls  of the latter interpretation, as b) and c) are
> likely to be another  (subjective) judgment call.
>
> Thus it seems to make sense to "charter carefully because
> you've got to live with the results", and "charter with
> sufficient number of checkpoints to check whether the WG is
> producing the right stuff".   However, this doesn't
> necessarily mean the chartering process needs to be slow..

---

From: Dave Crocker <[email-masked]>
Date: Sat, 14 Feb 2004 08:13:41 -0800

(many thanks to bernard for creating this line of discussion and John for
focusing on the startup issue.)

JCK>  ...  A decade ago, a BOF
JCK> (much less a couple of BOFs) were not requirements for creating 
JCK> a WG.

In fact, a common requirement for authorizing a BOF was that there
_already_ be an active mailing list.  If the IETF still believes that
the primary venue for work is the mailing list, then that is where the
primary work of _formation_ needs to take place.

I think that the desires for "give it a try and stop it if there is not
sufficient progress" can be satisfied entirely by a simple structure
that involves almost no extra IETF administrative or management effort.
In fact it will require _less_ effort than needed today:

     1. Anyone wanting to form a working group must first create a
     mailing list, and a web page that lists at least a mailing list and
     archive, and a candidate charter,

     2. The mailing list must show enough history of activity to assess
     a pattern, specifically one of productive IETF-like discussion, for
     the charter and for the actual work.

     3. The IETF should provide a "WG Formative Efforts" page and list
     any and all efforts that satisfy step 1.

     4. A BOF can be authorized after #2 is satisfied.

This gives efforts some visibility and allows them whatever startup
thrashing they need. When they get past that stage and start doing
productive, IETF types of work, they can be treated as a serious
candidate for IETF oversight.

I think that "relaxed chartering" has the very serious problem of
creating lots of management pain, for little or no gain.  It is too easy
to do the pre-IETF work in a pre-IETF environment.  Forcing IETF
management to be involved -- especially with killing unproductive
efforts -- is pretty much a lose-lose structure.

d/
--
 Dave Crocker <dcrocker-at-brandenburg-dot-com>
 Brandenburg InternetWorking <www.brandenburg.com>
 Sunnyvale, CA  USA <tel:+1.408.246.8253>

---

From: John C Klensin <[email-masked]>
Date: Sat, 14 Feb 2004 11:39:46 -0500

Dave,

I think your proposal and mine are largely compatible.  Let me 
try to explain that...

I think there should be a number of different ways of getting to 
a WG and that _any_ model through which all pre-WGs and WGs must 
go is an invitation to getting ourselves bogged down in 
procedural nonsense and rigidity rather than keeping the focus 
on getting the job done.   So, e.g., any of the following ought 
to be considered reasonable, depending on circumstances, AD/ 
IESG/ community perspective, etc...

	* As you suggest, demonstrate, by action and drafts,
	that there is work which is ready to be turned into a
	WG, with or without a BOF or two, depending on what is
	and is not demonstrated.
	
	* Hold a BOF or two and demonstrate that things have
	crystallized that way (I note that we used to have a
	fairly rigid "two BOFs and no more" rule, but that, now,
	we seem to be able to have two or three BOFs, a change
	of name and microscopic change of focus, a few more
	BOFs,... and that cannot often be a good sign).
	
	* Spin up an RG which eventually makes a WG
	recommendation.
	
	* Charter a group on a "short leash" basis as my
	comments suggested.

That isn't an exclusive list -- I imagine that there might be a 
half-dozen other options that would make sense occasionally -- 
and some of the above might be combined.   For example, no 
matter how a WG gets started, I would hope for significant AD 
oversight in the first six months or a year with rapid 
plug-pulling if it heads resolutely off into the weeds (on 
Valentine's Day, I am somehow drawn to analogies to couples who 
have really good relationships until they get married, then 
start having serious trouble almost immediately).

For me, the observation that some ADs have started using BOFs as 
a way to getting the community to shoot down a WG proposal, 
rather than doing it themselves, is a symptom of another 
problem.  But the solution isn't obviously to replace a 
BOF-rigidity with some other rigidity.  And there are certainly 
groups who could have, and did, managed to organize pre-WG (and 
even pre-BOF) mailing list discussions, documents, etc., which 
ended up not working out well as WGs.

In these troubled times, there is also two risks inherent in 
lots of pre-BOF work: (i) if a small, semi-private, group holds 
a lot of discussions and formulates a position, any IETF review, 
even if it occurs the day after chartering, is likely to be 
"late" review in terms of its interactions with already-hardened 
positions.  (ii) And such discussions may not be subject to IETF 
IPR provisions which could lead to some interesting ways to 
plant time bombs.  We could protect against some of that by 
being aggressive about the "formative efforts" listing you 
suggest, but such a listing would be taken by many groups (and 
their publicists) as IETF endorsement of, and commitment to, 
their efforts.   So, while I think the model you suggest would 
be a very useful tool (and, as you say, we more or less used to 
do it that way), there are unfriendly dragons by the sides of 
the path and we need to be aware of them.

In no case do I think we can afford to get ourselves into a "if 
you manage to jump through the following hoops, you are entitled 
to have a WG, and, if the WG jumps through the following 
additional hoops, it is entitled to get a standard approved". 
More flexibility about creation, evaluation, and termination 
would seem to help us avoid even the impression of that 
situation.

     regards,
       john

--On Saturday, 14 February, 2004 08:13 -0800 Dave Crocker 
<[email-masked]> wrote:

> (many thanks to bernard for creating this line of discussion
> and John for focusing on the startup issue.)
>
>
> JCK> (much less a couple of BOFs) were not requirements for
> creating  JCK> a WG.
>
> In fact, a common requirement for authorizing a BOF was that
> there _already_ be an active mailing list.  If the IETF still
> believes that the primary venue for work is the mailing list,
> then that is where the primary work of _formation_ needs to
> take place.
>
> I think that the desires for "give it a try and stop it if
> there is not sufficient progress" can be satisfied entirely by
> a simple structure that involves almost no extra IETF
> administrative or management effort. In fact it will require
> _less_ effort than needed today:
>
>
>      1. Anyone wanting to form a working group must first
> create a      mailing list, and a web page that lists at least
> a mailing list and      archive, and a candidate charter,
>
>      2. The mailing list must show enough history of activity
> to assess      a pattern, specifically one of productive
> IETF-like discussion, for      the charter and for the actual
> work.
>
>      3. The IETF should provide a "WG Formative Efforts" page
> and list      any and all efforts that satisfy step 1.
>
> This gives efforts some visibility and allows them whatever
> startup thrashing they need. When they get past that stage and
> start doing productive, IETF types of work, they can be
> treated as a serious candidate for IETF oversight.
>
> I think that "relaxed chartering" has the very serious problem
> of creating lots of management pain, for little or no gain.
> It is too easy to do the pre-IETF work in a pre-IETF
> environment.  Forcing IETF management to be involved --
> especially with killing unproductive efforts -- is pretty much
> a lose-lose structure.
>
>
> d/
> --

---

From: Dave Crocker <[email-masked]>
Date: Sat, 14 Feb 2004 09:48:46 -0800

JCK> I think your proposal and mine are largely compatible.

oh, you optimist, you...

JCK> I think there should be a number of different ways of getting to
JCK> a WG and that _any_ model through which all pre-WGs and WGs must 
JCK> go is an invitation to getting ourselves bogged down in 
JCK> procedural nonsense and rigidity rather than keeping the focus 
JCK> on getting the job done.

I think we have a meaningful difference, here.

By their nature, rules are Procrustean.  Hence, rules should be created
only when the tradeoffs among flexibility, efficiency, quality
and the like all warrant it.  In this case, I think we have a very
serious problem with wasteful formation of unproductive working groups.

So, the rule is a very straightforward way of moving the cost of
wastefulness to be outside of IETF concern. It creates a simple and
entirely relevant admission criterion: To get chartered for doing IETF
work, a group much demonstrate that it is already _able_ to do IETF work.

Of the alternatives you list, some are in fact not so good.  For example, an
IRTF activity may or may not be productive as an IETF effort, so I'm not
inclined to use the IRTF history as an automatic ticket to working group
status.  Some IRTF efforts do show the necessary qualities, but others
do not. Dealing with this is simple: subject it to the same requirements
as all other efforts.  For a good effort, the previous IRTF effort will
make satisfying the criteria I list trivial.

JCK> That isn't an exclusive list -- I imagine that there might be a
JCK> half-dozen other options that would make sense occasionally --

Fuzzy procedures and criteria invite sloppy, wasteful action. Since this
is what we have been seeing regularly, in working group formation, we
need to make sure that an effort to fix the problem really does fix it.

We should only make rules that respond to a demonstrated need, but the
rules should be as simple and clear and precise as we can make them.

As Bert notes, in reality I'm only documenting an established good
practise.  The problem has been not enough adherence to it.

And let's be clear about the 'rigidity' that I am proposing.  Note that
the actual assessment of productivity is unspecified.  _That_ is where
it makes sense to use the typically fuzzy rough-consensus approach, not
in the basic procedure.

JCK> In these troubled times, there is also two risks inherent in
JCK> lots of pre-BOF work: (i) if a small, semi-private, group holds 
JCK> a lot of discussions and formulates a position, any IETF review, 
JCK> even if it occurs the day after chartering, is likely to be 
JCK> "late" review in terms of its interactions with already-hardened

First of all, the requirement I offered for pre-IETF work is that it be
conducted in an IETF manner.  This means open and flexible, as well as
productive.

Second of all, we already have that issue in many situations, which
means we already have ways of dealing the problem you list. So, either a
group that wishes IETF assistance and imprimateur is willing to conduct
itself according to IETF requirements, or it isn't.

JCK> positions.  (ii) And such discussions may not be subject to IETF 
JCK> IPR provisions which could lead to some interesting ways to 
JCK> plant time bombs.

We are stuck with that reality, no matter what.

JCK>   We could protect against some of that by 
JCK> being aggressive about the "formative efforts" listing you 
JCK> suggest, but such a listing would be taken by many groups (and 
JCK> their publicists) as IETF endorsement of

John, perhaps you have noticed that _any_ action is subject to political
exploitation. The only way to avoid such exploitation is to be invisible
and inactive.

We simply cannot let fear of external stupidity and abuse dominate our
thinking and working.

If we think something is reasonable and productive, in its own right,
then it is what we should do. If others choose to be stupid or nasty or
exploitive about it, that is their problem and the problem of anyone who
believes them.

JCK> In no case do I think we can afford to get ourselves into a "if
JCK> you manage to jump through the following hoops, you are entitled 
JCK> to have a WG,

I agree completely.  My suggestion was for steps that should be
necessary.  But, no, they are not sufficient by themselves.

They are designed to provide highly relevant input to the formation
decision process, not to replace that process.

and now for something completely different...

WBB> Interestingly, those two questions and actions is what many (if not all) ADs
WBB> ask and want to see before approving a BOF. Some people do not like that...
WBB> but... such is life.
WBB> So it seems we are doing the right thing?

Bert, Your note triggers two lines of reaction from me:

1. Established Practise

If the IESG were typically imposing the requirements I listed, then we
would not be experiencing wasteful BOFs and wasteful working group
startup "thrashing". Since we do, in fact, _frequently_ experience both
of these problems, then no, it is not what all (or possibly even most)
ADs do.

Some do, yes.

In fact, my intent was merely to document and require practise that I
have viewed as long-used and effective. The problem has been with its
inconsistent application.

2. Response from an AD

Since I have not yet done IETF work that interacted with you, I am
intrigued by your note. My lack of first-person experience with you is
tempered by extremely consistent comments from others that you are an
excellent AD.

From what I can determine, you in particular have nothing to be
defensive about. Yet I take your note frankly to be defensive.

(I apologize for making this personal, but I do not know any other way
to broach this specific topic and I think the topic of IESG
participation in these discussions is absolutely fundamental to the
question of making improvements in IETF timeliness and productivity. My
own view is that John K's trust/distrust proposals go at exactly this
issue. So my response, here, is really a plea: We must find a way to
have IESG members engaged in these discussions without our attacking
them and with their being candid with us. Defensive or hostile tone or
content are simply not productive.)

Clearly "we are doing the right thing" is not correct. "We" the IESG, and
"We" the IETF, frequently are _not_ doing the right thing.  "Some", yes.
"All", no.

In particular, "not enough".  Not nearly enough.

The IETF has established a very damaging pattern with working group
formation, conduct and output. All three are highly problematic. Not
every working group, of course. But enough to prompt the crisis that we
have been trying to address for two years. Yet those two years have
produced virtually no significant progress in improving any of them.

d/

---

From: John C Klensin <[email-masked]>
Date: Sat, 14 Feb 2004 17:30:57 -0500

> JCK> I think your proposal and mine are largely compatible.

Sigh.

> JCK> I think there should be a number of different ways of
> getting to JCK> a WG and that _any_ model through which all
> pre-WGs and WGs must  JCK> go is an invitation to getting
> ourselves bogged down in  JCK> procedural nonsense and
> rigidity rather than keeping the focus  JCK> on getting the
> job done.
>
> By their nature, rules are Procrustean.  Hence, rules should
> be created only when the tradeoffs among flexibility,
> efficiency, quality and the like all warrant it.  In this
> case, I think we have a very serious problem with wasteful
> formation of unproductive working groups.

Our diagnosis differs somewhat.   I'd say we have a lifecycle 
problem with the time between when an effort starts to come 
together and when we either have it producing standards or move 
it off the IETF queue. While we can certainly manipulate things 
like nominal starting points to make the IETF look better 
statistically, I don't consider that a useful objective.  There 
are perfectly good efforts that don't belong in the IETF but 
that need some input and feedback from us.  We may be able to 
figure that out quickly; it may take a while.  Leaving them on 
their own for a long time and then providing them that feedback 
when they decide they want a charter may be the right thing to 
do for some of them, early BOFs and strong early feedback may be 
right for others, and the spotlight of a WG which then gets its 
plug pulled may be right for yet others.  I think we have a 
serious problem in that we are wasteful in the processes we use 
to [decide to] create and terminate WGs.    I think your 
conclusion, above, is a symptom of the fact that we have made 
the creation and deletion processes much too complex and 
ponderous.

So, on the one hand, were I giving ADs advice, my advice would 
be that, in a very large fraction of cases, the model you and 
Bert have suggested is the right one.   But I'd like to see ADs 
look at these situations one at a time, make judgments 
(consulting mailing lists, directorates, and other mechanisms 
for understanding community views as appropriate), and then 
choose models, tools, and methods for particular groups as they 
think appropriate... to the situation and to their particular 
management styles and preferences.

Obviously, "trust the IESG" models are an important part of that 
approach.   If those folks can't figure out which pre-WG (or 
during-WG, or post-WG) approach is likely to work best for them 
and a given situation, I don't see any reason to believe that 
they can do better with a single standardized sequence of events 
that still cumulate in the need to make subjective evaluations 
and decisions. We also got some new data yesterday that, I 
believe, deserves very careful consideration: for all of the 
complaints about IESG poor decisions, poor allocations of 
priorities, and so forth, the Nomcom looked at the IESG, 
discussed issues with the community, evaluated people and 
alternatives... and returned every single IESG member except the 
one who had voluntarily stepped down.  Like it or not, that is a 
pretty strong statement, on behalf of the community, that we are 
in reasonable shape and that IESG members are behaving 
reasonably and making mostly the right decisions.  If we don't 
believe that conclusion, we need to stop wasting our time with 
little details like WG creating and, instead, start thinking 
about how to replace the Nomcom model and with what.  And, if we 
do believe it, then, IMO, the IESG may need suggestions, 
guidelines, and reminders, but probably doesn't need more rule 
to try to constrain their behavior (especially since rules to 
constrain them have a history of being, effectively, ignored).

I think the rest of your note reflects this difference in 
perspective, so won't make this one longer by running through it 
point by point.   I do want to note that nothing I suggested 
involves "an automatic ticket to WG status".   Each of them, 
including, to me, your suggestion, is a way for the IESG to 
accumulate data on which a decision can then be made -- a 
decision to establish a WG, to send the group elsewhere, or to 
advise them to develop more data (using the same or different 
methods).

    john

---

From: Dave Crocker <[email-masked]>
Date: Sat, 14 Feb 2004 14:58:40 -0800

JCK>   While we can certainly manipulate things
JCK> like nominal starting points to make the IETF look better 
JCK> statistically, I don't consider that a useful objective.

Neither do I.

There are two purposes to my propsal:

1. Impose a rigorous barrier to IETF entry

It will filter out quite a
bit of poor work, simply by requiring that a group demonstrate that it
can be productive, _before_ it is made a working group. This is an
entirely natural barrier.

Do we have significant indication that working groups with truly lousy
initial behavior later turn out to do something valuable? (I'm looking
for a pattern here, not an exception.) If a working group cannot have
its act together initially, it is not ready for open, standards-oriented
engineering prime time. So let's not give them a ticket to the game
until they are ready.

Please note that I did not say to ignore them until they are chartered.
The suggestion to list them gives them visibility.  I've no doubt some
other cheap and useful actions will also help.

2. Significantly reduce the IETF management and operations burden of
wasteful working groups.

This is not just an AD issue. There is also the small matter of WG chair
time, Secretariat time, IETF meeting space and time, etc. All of these
are very, very scarce resources, which we have a pattern of squandering.

Rather than task ADs with watching out for questionable working groups
and rather than having those questionable working groups add to the
congestion of IETF week meeting time, move the startup noise out of the
organization.

JCK>  There 
JCK> are perfectly good efforts that don't belong in the IETF but 
JCK> that need some input and feedback from us.  We may be able to 
JCK> figure that out quickly; it may take a while.

That sounds as if the IETF has some sort of track record "taking awhile"
and giving corrective feedback that is productive.  Again, I'd be
interested in hearing about the pattern of achievement here, because my
own sense is that it is quite poor.

JCK> So, on the one hand, were I giving ADs advice, my advice would
JCK> be that, in a very large fraction of cases, the model you and 
JCK> Bert have suggested is the right one.   But I'd like to see ADs 
JCK> look at these situations one at a time, make judgments 
JCK> (consulting mailing lists, directorates, and other mechanisms 
JCK> for understanding community views as appropriate), and then 
JCK> choose models, tools, and methods for particular groups as they 
JCK> think appropriate... to the situation and to their particular 
JCK> management styles and preferences.

In a perfect world, I would agree with you.  In world with ADs who were
consistently expert in these skills and in a world where ADs had plenty
of time for such activities, I would agree with you.

However the reality is that the IETF succeeds when a group self-form and
had develops enough motivation and cohesion to be productive. For any
other scenario, I believe the IETF experience is frustrating, at best,
and more often wasteful and unproductive.

The idea that ADs actually can or should have the task of "teaching"
working groups to be productive is a very serious and strategic
management error, based on the IETF performance I've witnessed.

JCK>    I do want to note that nothing I suggested
JCK> involves "an automatic ticket to WG status".

Yeah, I thought you'd take exception to that, and indeed it went beyond
your words.

However it is not clear what to do with something like "prior IRTF
effort", beyond feed it into the paradigm I am suggesting. Once we try
to do anything else with it, I believe the pragmatics turn it into
"automatic ticket". Anything else, I believe, will be too subtle for
practical use.

d/

---

From: Pete Resnick <[email-masked]>
Date: Sat, 14 Feb 2004 17:38:08 -0600

On 2/14/04 at 2:58 PM -0800, Dave Crocker wrote:

>It will filter out quite a bit of poor work, simply by requiring 
>that a group demonstrate that it can be productive, _before_ it is 
>made a working group. This is an entirely natural barrier.
>
>Do we have significant indication that working groups with truly 
>lousy initial behavior later turn out to do something valuable? (I'm 
>looking for a pattern here, not an exception.) If a working group 
>cannot have its act together initially, it is not ready for open, 
>standards-oriented engineering prime time. So let's not give them a 
>ticket to the game until they are ready.
>
>Please note that I did not say to ignore them until they are 
>chartered. The suggestion to list them gives them visibility. I've 
>no doubt some other cheap and useful actions will also help.

I'm trying really hard to figure out what exactly is gained by not 
chartering a group but still letting them start their work, put them 
on a list, and expect the IETF-writ-large to start monitoring and/or 
working with them. If the IETF-writ-large is going to have to 
participate anyway (i.e., the folks in the not-yet-working-group 
request and get early cross-area feedback from other folks in the 
IETF before they produce their first I-D), that sounds like using 
exactly the same resources that a working group would. If the 
IETF-writ-large is *not* going to participate, there is some great 
likelihood that the group will not get good cross-area review, they 
will float off into the weeds, and they will have invested a great 
deal of effort in what will turn out to be useless work (i.e., the 
"late surprise"/"not enough early feedback" problem), which I thought 
was exactly one of the problems we were trying to fix.

(Note that I have not made a distinction here between "outsiders" who 
want a working group and IETFers who want a working group. There is 
now a significant enough number of people inside the IETF who are 
perfectly capable of doing lots of bad work if left to their own 
devices.)

If we're going to ask folks to demonstrate "good work", that's going 
to include:

- getting architectural input, security input, transport input (e.g., 
if your working in applications), applications input (e.g., if you're 
working down in the lower layers), etc.
- having a well-run mailing list with good moderation (i.e., a chair)
- producing I-Ds.

Once we add putting them on a public list or other "visibility 
mechanisms" (maybe having a document stating what they intend to do, 
i.e., a charter), what do we have that distinguishes that from a 
working group? If the answer is, "We don't have to spend weeks having 
the IESG argue about the charter, and the default action if they do 
bad work after a while is that they go away", why don't we just tell 
the IESG that they should stop arguing about charters and that bad 
working groups should be killed early and often?

pr
-- 
Pete Resnick <http://www.qualcomm.com/~presnick/>
QUALCOMM Incorporated - Direct phone: (858)651-4478, Fax: (858)651-1102

---

From: Dave Crocker <[email-masked]>
Date: Sun, 15 Feb 2004 07:55:30 -0800

Pete,

PR> I'm trying really hard to figure out what exactly is gained by not
PR> chartering a group but still letting them start their work, put them 
PR> on a list, and expect the IETF-writ-large to start monitoring and/or 
PR> working with them.

The difference is quite large. I apologize for not succeeding at making
it clear enough.

First of all, there is no "letting them start".  That's a key point.
They must take their own initiative and self-form a working entity.

I need to stress that this is what already happens for efforts that have
real, community-based impetus. So what would be different?  We
would not be expending resources on efforts that try to become an IETF
working group and only _then_ figure out what the heck they are going to
do.

Second of all, the administrative and management overhead on the IETF is
almost zero, which cannot be said for chartered working groups.  And
they get no IETF meeting time.

Since a mailing list is supposed to be the core of working group effort,
we should make the ability to operate over a mailing list a
pre-condition for working group status.

R>  they
PR> will float off into the weeds, and they will have invested a great 
PR> deal of effort in what will turn out to be useless work (i.e., the 
PR> "late surprise"/"not enough early feedback" problem), which I thought 
PR> was exactly one of the problems we were trying to fix.

Well, this is certainly an important concern.  So let's be clear about
the suggestion:  It's not that the group should seek IETF formal status
when it is nearly done.  It should seek formal status when it has
achieved enough coherence to show motivation and focus.  For any good
group, that is usually very quickly.

PR> If we're going to ask folks to demonstrate "good work", that's going
PR> to include:

First of all, there is a difference in the IETF's leverage between a
wg-in-formation, versus an already-chartered working group. The former
is (or should be) highly motivated to take the initiative of getting
itself on a productive vector. The degree to which the latter has to
develop that motivation or must be pressured towards it by their AD has
proven consistently problematic.

Second of all demonstrating the motivation and "skill", at recruiting
whatever input is needed, is a key point. Better to have them fail at
doing that before they are working group than after.

PR> Once we add putting them on a public list or other "visibility
PR> mechanisms" (maybe having a document stating what they intend to do, 
PR> i.e., a charter), what do we have that distinguishes that from a 
PR> working group? If the answer is, "We don't have to spend weeks having 
PR> the IESG argue about the charter, and the default action if they do 
PR> bad work after a while is that they go away", why don't we just tell 
PR> the IESG that they should stop arguing about charters and that bad 
PR> working groups should be killed early and often?

The footprint of IETF impact is larger than that, as I've noted above
and earlier. But the issue is larger than that.

IETF Working Groups succeed when they have coherence and focus. The
ability to form into a coherent group and develop a discussion process
with a productive vector should be a fundamental test for the creation
of a working group.

If folks cannot demonstrate these, they are not ready for IETF prime
time.

So let's make these a pre-charter requirement, rather than do the
charter and spend considerable resources trying to "fix" groups with
this problem.

d/

---

From: Pete Resnick <[email-masked]>
Date: Sun, 15 Feb 2004 10:44:51 -0600

On 2/15/04 at 7:55 AM -0800, Dave Crocker wrote:

>>chartering a group but still letting them start their work, put 
>>them on a list, and expect the IETF-writ-large to start monitoring 
>>and/or working with them.
>
>The difference is quite large. I apologize for not succeeding at 
>making it clear enough.

Maybe I'm being dense, but this message didn't really clear up the 
situation for me. Let me try to ask specific questions.

>First of all, there is no "letting them start".

That I understood. I didn't mean to imply that "letting them start" 
was an active action by the IETF.

>We would not be expending resources on efforts that try to become an 
>IETF working group and only _then_ figure out what the heck they are 
>going to do.

OK, so here's the first one: What resources? Are we talking about all 
that effort that now goes on during charter review? Well, a different 
answer to that is, "Stop spending all of that effort doing 
nit-picking charter reviews." We seem to be doing that so that we can 
have some grand justification to stop working groups from doing bad 
things in the future. There's another tool at our disposal to do 
that, which is to have the AD mercilessly kill the working group if 
it strays.

If you're not talking about charter review, what resources are you 
talking about?

>Second of all, the administrative and management overhead on the 
>IETF is almost zero, which cannot be said for chartered working 
>groups.

What overhead are we talking about here? Secretariat time? IESG time? 
They're certainly going to use the same technical review overhead as 
a chartered working group if they're doing the right thing to get 
chartered in the future, and that seems like the biggest resource 
use. What overhead are you referring to specifically?

>And they get no IETF meeting time.

Neither do working groups if an AD doesn't want them to.

>Since a mailing list is supposed to be the core of working group 
>effort, we should make the ability to operate over a mailing list a 

That I understand.

>>they will float off into the weeds, and they will have invested a 
>>great deal of effort in what will turn out to be useless work 
>>(i.e., the "late surprise"/"not enough early feedback" problem), 
>>which I thought was exactly one of the problems we were trying to 
>>fix.
>
>Well, this is certainly an important concern.  So let's be clear 
>about the suggestion:  It's not that the group should seek IETF 
>formal status when it is nearly done.  It should seek formal status 
>when it has achieved enough coherence to show motivation and focus. 
>For any good group, that is usually very quickly.

OK, but this is starting to sound like, "Form a mailing list and 
demonstrate on that mailing list that you have a clue." That doesn't 
require producing documents or doing a bunch of work prior to 
chartering; it's just demonstrating some ability to have a coherent 
discussion. But do we have lots of examples now of working groups 
that are chartered which have not been having some coherent 
discussions on *some* mailing list, even if that mailing list was not 
dedicated to a particular activity? If a bunch of folks start having 
a coherent discussion on the IETF-SMTP list about some wiz-bang 
extension to SMTP and get a charter together, is there really a need 
to have them go off and have some number of weeks of discussion on a 
new mailing list before we charter the group? Or if an IRTF group 
comes up with a good engineering idea, a proposal for what needs to 
be done, and a group of people on the IRTF list that commit to doing 
the work in the IETF, do we really need another hoop for them to jump 
through?

>there is a difference in the IETF's leverage between a 
>wg-in-formation, versus an already-chartered working group. The 
>former is (or should be) highly motivated to take the initiative of 
>getting itself on a productive vector. The degree to which the 
>latter has to develop that motivation or must be pressured towards 
>it by their AD has proven consistently problematic.

But that sounds like more of a problem with ADs being unwilling to 
pull the rip cord early enough on unmotivated working groups instead 
"pressuring" them to do something, not with the chartering process.

pr

---

From: Dave Crocker <[email-masked]>
Date: Sun, 15 Feb 2004 10:00:30 -0800

Pete,

PR> that effort that now goes on during charter review? Well, a different
PR> answer to that is, "Stop spending all of that effort doing 
PR> nit-picking charter reviews." We seem to be doing that so that we can 
PR> have some grand justification to stop working groups from doing bad 
PR> things in the future. There's another tool at our disposal to do 
PR> that, which is to have the AD mercilessly kill the working group if 
PR> it strays.

I think the difference in our views involves very different expectations
about changing people's behavior.

We have a track record of people behaving a certain way that is
inefficient and unproductive. Lots of different people, over a long
period of time. So it clearly is not a matter of particular people. It
is a matter of the organizational structure and process. It is a matter
of context. Yes, people can change, but we have not yet figured out how
to do that in this forum.

Rather than mandate that various people shall start behaving
differently, the suggestion is to remove the problem from the IETF
management venue.  Allow entry only after a group has demonstrated a
basic ability to do the work necessary.

In other words, the problem is with the context, so let's change the
context.

d/

---

From: John C Klensin <[email-masked]>
Date: Sun, 15 Feb 2004 16:47:45 -0500

--On Sunday, 15 February, 2004 10:00 -0800 Dave Crocker 
<[email-masked]> wrote:

> I think the difference in our views involves very different
> expectations about changing people's behavior.
>
> inefficient and unproductive. Lots of different people, over a
> long period of time. So it clearly is not a matter of
> particular people. It is a matter of the organizational
> structure and process. It is a matter of context. Yes, people
> can change, but we have not yet figured out how to do that in
> this forum.
>
> differently, the suggestion is to remove the problem from the
> IETF management venue.  Allow entry only after a group has
> demonstrated a basic ability to do the work necessary.
>
> In other words, the problem is with the context, so let's
> change the context.

Dave,

One problem I have with your suggestion -- I think the major one 
-- is that it would actually reduce our ability to say "no, go 
away, this doesn't belong here" in a prompt and efficient way. 
As Pete points out, it would save surprisingly little in terms 
of the other things we do.

Can that be fixed by "changing the context"?  Not, I think, in 
the way you suggest.  If we stop the circumlocutions, I think 
you are saying "the IESG doesn't have what it takes to shut down 
WGs that are not being productive".  I think that is a problem, 
a big one.  But, if you believe that can't be changed, then why 
is it reasonable to believe that they won't permit WGs to be 
created with half-baked evidence of viability and then permit 
_those_ to go on forever?  I'd rather see us

	* Remind those ADs who haven't gotten Bert's message
	that getting a group to demonstrate that it is
	functional is usually a good pre-BOF criterion.
	
	* Focus as much thinking as possible on the principle
	that some WGs succeed in spite of sloppy charters, and
	others fail in spite of huge amounts of time spent
	trying to get their charters exactly right, and that
	this suggests we ought to be spending more time on
	technical work, and monitoring of technical work, than
	on charter nit-picking (regardless of what does, or does
	not, precede chartering).
	
	* Make sure the IESG understands that they will have
	strong community backing for killing off WGs that don't
	appear to be functional and/or worth the resources they
	consume.  My strong impression is that, in the past, the
	perceived lack of that backing has been an important
	contributor to the problem of letting these groups
	drift: often, when one is shut down, the people who
	still care about its work raise a mighty outcry, while
	the rest of the community ignores the situation.  That
	makes it, for most ADs, a lot easier to let a moribund
	group drag on -- at least to the point that its
	ineffectualness is painfully obvious to everyone -- than
	to shut it down when that would be most efficient.
	This is another situation in which, IMO, the community
	keeps sending the wrong message to the IESG and the IESG
	does what we want them to do, which is to listen and
	respond.

The procedural change we may have a place for in this is a WG 
with a specified sunset time.   E.g., a "Here is a charter, you 
are now subject to all of the normal WG rules, but you have X 
months to demonstrate that you are worth the trouble.   If you 
can demonstrate within that period that you are functional, we 
can negotiate a new charter.  If not, you disappear".  I can't 
imagine a value of X greater than 12 months would ever make 
sense, and I can see values tied to an IETF meeting or two 
(e.g., "you get to meet at IETF NN, but, if you haven't shown 
significant progress by the time scheduling comes around for 
IETF NN+1, not only do you not get a meeting slot, but you 
disappear entirely").

More speedy and efficient experiments within the system, 
followed by quick termination if the experiments _succeed_ in 
showing that the ducks aren't lined of or the topic is 
IETF-inappropriate.  But not more ways to encourage people to 
work in a way that is sort of in the system, and sort of out, 
with the expectation of monitoring that busy people won't do, 
etc.

Pete has covered most of the rest of what I would say here if he 
hadn't, and expressed it better than I would have.

    john

---

From: Dave Crocker <[email-masked]>
Date: Sat, 21 Feb 2004 22:41:29 +0900

John,

JCK> One problem I have with your suggestion -- I think the major one
JCK> -- is that it would actually reduce our ability to say "no, go 
JCK> away, this doesn't belong here" in a prompt and efficient way.

It does not do that at all.  I think, perhaps, you've taken my
suggestion as meaning that performing the steps I described would be
"sufficient" for gaining working group status.

My suggestion is not meant to remove any other criteria that we already
have.

And let me again stress that my suggestion is nothing but codifying
existing practise, in or to have it be applied consistently, as Bert
has essentially pointed out.

JCK> As Pete points out, it would save surprisingly little in terms 
JCK> of the other things we do.

I'm frankly surprised that the difference in management and
administrative overhead seems such a small thing to you.

JCK> Can that be fixed by "changing the context"?  Not, I think, in 
JCK> the way you suggest.  If we stop the circumlocutions, I think 
JCK> you are saying "the IESG doesn't have what it takes to shut down 
JCK> WGs that are not being productive".

The problem is bigger and deeper than that.  Working group creation is a
particular moment of incentives and leverage. After chartering, the
only real leverage is changing chairs or disbanding the working group.

First of all, those are big hammers.  Absent smaller ones then we
should do as much as we can to make sure the big hammer is not needed.
More stringent "entrance requirements" is a pretty obvious way to make
sure we are getting better "students" of the IETF process.

As for the IESG track record, you are kidding, right?  Do we have a
solid pattern of wasteful, thrashing working groups that drag on,
being disbanded?  I could have sworn we did not.

That leaves the IESG question to one of "they will do better". And
that leads to the questions "why, how and when"?

Your suggestion requires no procedural changes.  So the IESG could
have started taking this more stringent position anytime.  Yet they
haven't.

JCK>   I think that is a problem, 
JCK> a big one.  But, if you believe that can't be changed, then why 
JCK> is it reasonable to believe that they won't permit WGs to be 
JCK> created with half-baked evidence of viability and then permit 
JCK> _those_ to go on forever?  I'd rather see us

Your question is particularly important given that any reasonable
evaluation of the IETF's track record would say that, indeed, that is
exactly what has been happening.

The idea is to change the IETF culture about the requirement, so that
it is a community thing, rather than an IESG thing.  The idea is to
have the high bar for entrance be a shared value, with complete
transparency.

JCK> 	this suggests we ought to be spending more time on
JCK> 	technical work, and monitoring of technical work, than
JCK> 	on charter nit-picking (regardless of what does, or does
JCK> 	not, precede chartering).

If a group cannot formulate a clear, focused problem statement and a
clear, focused, practical statement of the solution path, then
concern over the charter is not nit-picking.  It means that the group
really has no idea what it is going to do.

This being an engineering group rather than a research group or coffee
klatsch, it is rather serious for them to lack a shared sense of the
problem and how to approach solving it.

JCK> 	* Make sure the IESG understands that they will have
JCK> 	strong community backing for killing off WGs that don't
JCK> 	appear to be functional and/or worth the resources they
JCK> 	consume.

So, rather than put the affirmative burden on the group supposed to do the work
you want to increase the responsibility on the IESG to perform a
negative action. This seems like a very bad process design.

JCK>  My strong impression is that, in the past, the
JCK> 	perceived lack of that backing has been an important
JCK> 	contributor to the problem of letting these groups

My strong impression is that that your strong impression is crap.

First of all, working groups thrash for a reason. Threats are rarely
successful against that reason. So the "simple" step of somehow giving
ADs more resolve entirely ignores the core problem. Feel free to cite
a pattern that shows otherwise.

Second of all, my strong impression is that ADs who are diligent and
forceful about shutting down wayward working groups do just fine.

Third of all, the model of expecting people who are not senior
management in their day job to display the kind of management
understanding and skill this requires is just plain inappropriate.

And lastly, relying on ADs to do post-hoc quality control is entirely
the wrong model.  Note the constant return to the question of
"trusting" ADs.  The IETF's problems are not really about ADs.  They
are about working groups doing their work.  So if we are going to make
changes, let us try to make sure that they change how working groups
work.

Once again:  You are seeking to change a possible community failure to
give proper support for ADs who might otherwise get the resolve to
shut down bad working groups.  I hope both the fuzziness and
indirectness of your model is clear in that description.

By contrast, I am merely suggesting that we require a group that is
going to consume IETF resources demonstrate that it has the skills
necessary to do the job it is contracting for.

JCK> 	drift: often, when one is shut down, the people who
JCK> 	still care about its work raise a mighty outcry, while
JCK> 	the rest of the community ignores the situation.

Of course they raise a mighty cry.  That does not mean anything.  Have
we suddenly become afraid of noise?

What means something is an appeal and a reversal.  Do we have a track
record of shutdowns being reversed?  Do we have a track record of
groups that are shut down later doing well (presumably elsewhere)?

JCK>   That
JCK> 	makes it, for most ADs, a lot easier to let a moribund
JCK> 	group drag on

Now you are at the heart of the matter:  Shutting down a group is a
painful act.  That's why we need to make it a true last-resort.  What
you want to do is to make it a primary management tool.

JCK> -- at least to the point that its
JCK> 	ineffectualness is painfully obvious to everyone

I might have thought that, too, but then we get groups like IMPP that
went through 4 years, 3(!) sets of working group chair, and still did
not have its act together.

JCK> The procedural change we may have a place for in this is a WG
JCK> with a specified sunset time.   E.g., a "Here is a charter, you 
JCK> are now subject to all of the normal WG rules, but you have X 
JCK> months to demonstrate that you are worth the trouble.   If you 
JCK> can demonstrate within that period that you are functional, we 
JCK> can negotiate a new charter.

Given our limited resources, why in the world would this sort of "say
yes easily and then try to figure things out later" model work?

Where can we look for an example that shows a pattern of success with
such a model?

d/

---

From: John C Klensin <[email-masked]>
Date: Sun, 22 Feb 2004 14:20:04 -0500

--On Saturday, 21 February, 2004 22:41 +0900 Dave Crocker
<[email-masked]> wrote:

> John,
> 
> 
> JCK> One problem I have with your suggestion -- I think the
> major one JCK> -- is that it would actually reduce our ability
> to say "no, go  JCK> away, this doesn't belong here" in a
> prompt and efficient way.
> 
> suggestion as meaning that performing the steps I described
> would be "sufficient" for gaining working group status.

No, I think it does something a little different.   If we use,
more or less, the current processes (see below), an AD who has
told a group to demonstrate that it can be productive can say,
at almost any time during the pre-WG, pre-BOF proof of concept
period "sorry, this doesn't belong in the IETF, why don't you
find another venue".  If my guess as to how what you have
proposed would play itself out is correct, we would be creating
another "late surprise" situation: a group would go off and
spend a lot of energy trying to prove concern, come in, and be
told, "even if you have demonstrated that you can do that, we
don't want it".  Yes, that helps the IETF statistically -- the
time that is wasted isn't "ours" -- but it isn't good for the
Internet, since some of these activities really could thrive in
other venues.

> My suggestion is not meant to remove any other criteria that
> we already have.
> 
> And let me again stress that my suggestion is nothing but
> codifying existing practise, in or to have it be applied
> consistently, as Bert has essentially pointed out.

No, it takes an existing practice that is used often, sometimes
with great success, and turns it into the _only_ option.   That
ignores and excludes other paths that might work better in
particular situations.   

As others have pointed out, if it were the existing practice,
and it was working well in the overwhelming number of cases, we
wouldn't be having this discussion.  Or we shouldn't be having
it and should, instead, be focusing on the bad practices and/or
stupidity of selected ADs who can't recognize a good thing when
they see it.  But, as far as I can tell, the existing practice,
even when applied, isn't good enough to solve a significant
fraction of the situations we are talking about.  

To draw a perhaps-unfortunate analogy, this suggestion has the
ring of mandatory sentencing laws: if one commits a crime that
meets certain criteria, then the exact penalty is specified by
law, with no possible deviations.  Such things have the
attraction that similar or identical crimes receive identical
punishment, making the system very predictable.  But they fail
to allow for special or unusual circumstances that the criteria
are, inevitably, never good (or prescient) enough to capture.
My own preference --for both sentences and WG creation -- is for
strong guidelines, perhaps even strong enough that judges or ADs
who go outside them are expected to explain their reasoning in
public, but for the flexibility for those people to consider all
of the circumstances of a particular situation and apply
judgment.

> JCK> As Pete points out, it would save surprisingly little in
> terms  JCK> of the other things we do.
> 

That is because I'm not convinced there is much difference in
management and/or administrative overhead.  No matter how we
constitute these things, ADs often have a choice between "manage
now or manage later".  Under your plan, the "now" part becomes
just general oversight.   But where, I think, we may agree is
that WGs that have a clear focus, good understanding of what
they are about, and effective ways of working with the rest of
the IETF simply require much less active management than ones
that fail on one or more of those criteria.  On the principle
that early course corrections typically require fewer turns of
the wheel --and may be more successful-- than later ones, I'd
rather see ADs in a position to decide that spending some energy
on a group early on in the hope that they will run more smoothly
(and require significantly less management time) later is a good
tradeoff.   And, to some extent, your model, in order to make a
reduction in management and administrative time early in a WG's
life, discards the ability to make that sort of decision.

> JCK> Can that be fixed by "changing the context"?  Not, I
> think, in  JCK> the way you suggest.  If we stop the
> circumlocutions, I think  JCK> you are saying "the IESG
> doesn't have what it takes to shut down  JCK> WGs that are not
> being productive".
> 
> The problem is bigger and deeper than that.  Working group
> creation is a particular moment of incentives and leverage.
> After chartering, the only real leverage is changing chairs or
> disbanding the working group.
> 
> First of all, those are big hammers.  Absent smaller ones then
> we should do as much as we can to make sure the big hammer is
> not needed. More stringent "entrance requirements" is a pretty
> obvious way to make sure we are getting better "students" of
> the IETF process.

Well, part of what I've been arguing for is precisely to reduce
the amount of energy it takes to lift and swing those hammers.
We've seen perfectly good WGs that would meet any entrance
criteria one could identify go off in the weeds after a while,
and seen external developments create situations in which the WG
is just on the wrong track (even if it remains within charter
and current on benchmarks).  "The world has passed you by,
events have overtaken you, you are therefore irrelevant and
should go away" should be as reasonable a criterion for shutting
down a WG as anything else we have... and your proposal won't
help with it at all.   If we really wanted to shut down the
unproductive, ineffective, and irrelevant, we would institute a
probationary period for WGs in which each new one got, say, a
year.  At the end of that time, it would automatically come up
for rechartering, would need to demonstrate that it was useful,
productive, and not consuming more resources than its likely
results justified.  If it could not make that demonstration, it
would disappear.  Similar "sunset" limits on WG Chairs, etc.,
might not be a bad idea either.  

Now, at one level, a proposal like that is complementary to what
you are proposing, not in conflict with it.  But, in both cases,
giving the ADs some explicit discretion seems to me to be more
appropriate than trying to impose a particular management
mechanism and style.  

> As for the IESG track record, you are kidding, right?  Do we
> have a solid pattern of wasteful, thrashing working groups
> that drag on, being disbanded?  I could have sworn we did not.

We do not.  That is a problem.  Indeed, to my mind, a serious
one.  But imposing a rigid set of mechanisms on the IESG won't,
IMO, fix it, _given the track record_.  Instead...

	* The track record predicts that they will simply ignore
	such rules if they don't like them.   Net effect: much
	wasted time, many hard feelings in the community.   Now
	the community could start recalling people for ignoring
	written rules, but we don't have any track record of
	that either (partially, IMO, because of the "big hammer"
	problem and partially because we have all observed that
	pointing out to the Nomcom that some particular AD
	routinely ignores the rules doesn't reliably predict to
	that AD's being retired).  (That is, of course, why
	"plan B" makes removal mandatory if an AD is recalled
	for ignoring certain types of rules.)
	
	* We further reduce the pool of possible ADs to
	including only those who like the management style we
	are trying to impose.

I'd rather the Nomcom judge ADs on effectiveness (of which
rule-following ought to be a part, but adherence to one
particular management strategy should be secondary).

> That leaves the IESG question to one of "they will do better".
> And that leads to the questions "why, how and when"?
> 
> Your suggestion requires no procedural changes.  So the IESG
> could have started taking this more stringent position
> anytime.  Yet they haven't.

And I suggest that your suggestion, if approved (and remember
that the IESG would need to approve it, so the discussion may be
a waste of time if the IESG doesn't like the idea), would
probably just be ignored by the same IESG, or the same ADs, who
aren't inclined to do it today.

> Your question is particularly important given that any
> reasonable evaluation of the IETF's track record would say
> that, indeed, that is exactly what has been happening.
> 
> The idea is to change the IETF culture about the requirement,
> so that it is a community thing, rather than an IESG thing.
> The idea is to have the high bar for entrance be a shared
> value, with complete transparency.

And I believe in reasonably high bars for entrance _and_ in
performance/ effectiveness reviews and in sunset periods.  I
believe that the longer a WG hangs around, the more it should
need to do to justify its continued existence.  I believe  that
any IETF activity that is consuming resources should be required
to justify its continued existence in terms of the marginal
opportunity costs of those resources.  I just don't believe that
a rigid rule about what someone has to do before getting a BOF
or a WG --in minimum rules or in sequencing-- is going to help.  

Perhaps a partial example would be helpful in clarifying where I
think we disagree.  Suppose a group showed up tomorrow with a
clearly-defined problem, a smattering of experience IETF types
in its membership, and a theory about how they were going to
proceed.  Suppose the formation of the group came out of a
bar-BOF at some IETF meeting, or a discussion in the hall during
some conference or other non-IETF meeting.   And suppose they
think they can get something together and have it ready for
standardization in nine months.   I'd prefer to let them
approach an AD with a proposal and a "good enough" charter
draft, skip BOFs, go directly to a chartered WG with a minimum
of fuss, recruit more participants as part of Charter posting,
and go do their thing.  I'd like the AD to be able to say "ok,
I'll sign off on this given your promise that you can be
finished in nine months, but, if you are still around at the end
of month 12, I'm going to shut you down and you can start
justifying a more realistic group and schedule".   Now _that_
would reduce management and administrative time and investment.
Will we see it very often?  No, but we have seen a few very
close approximations.  And I would like to see those cases less
bogged down in bureaucracy rather than more so.

> So, rather than put the affirmative burden on the group
> supposed to do the work you want to increase the
> responsibility on the IESG to perform a negative action. This
> seems like a very bad process design.

Nope.  The burden is on the group either way.  But groups rarely
kill themselves, nor do they generally conclude that they aren't
ready for whatever the next procedural step is.  The IESG is
going to need to make decisions either way ... unless one makes
the process completely mechanical so that the necessary steps
and actions are also treated as sufficient ones.

     john