Subject: Re: [Rfced-future] [IAB] Welcome to the RFC Editor Future Development Program

From: John C Klensin <[email-masked]>
Date: Wed, 01 Apr 2020 13:46:57 -0400

--On Wednesday, April 1, 2020 00:11 +0300 Jari Arkko
<[email-masked]> wrote:

>> Should the program worry about the structure of the series or
>> should the program worry about putting in place someone who
>> could lead the community in worrying about the structure of
>> the series (among other things)?  

> Definitely the latter.

Jari,

I think it is not as simple as the dichotomy in Eliot's question
implies. 

I'd like to assume, perhaps incorrectly, that the "among other
things" in that question implies the kind of expertise in
publications and editorial strategy that were part of Heather's
job description.   If that is true, then getting such a person
in place is going to require a job description that is
sufficiently specific to allow recruiting the type of person we
want (and excluding others).  Stephen's comment about the
relationship to the RPC is actually a good example of that.
I'll respond to that suggestions separately to keep this one
from becoming long, but the answer, one way or the other,
changes the RSE role significantly enough to probably make some
people qualified under one conclusion and others qualified under
either.  Of course we could write a job description that said
"well, maybe this will change and be added or deleted from the
job", but my experience, including with people with whom we
discussed the job that was ultimately given to Heather, is such
as statement will cause otherwise-qualified possible applicants
to decide it is not a job that they want.

On the other hand, if the only role we care about is to "lead
the community in worrying...", then almost anyone in the IETF
with a history (and wide community recognition and respect) of
being able to lead efforts, foster consensus, and stay
personally balanced on the issues rather than, even
unintentionally, trying to drive conclusions one way or the
other could do the job -- no specialized technical publications
strategy background required.  As an obvious example, you would
be excellent if you were willing to take it on and had the time.

The other problem, of course, is that going very far in that
direction might mean we wouldn't have an RSE in place until the
middle of 2021, if we concluded we need an RSE at all.

best,
   john

---

From: "John Levine" <[email-masked]>
Date: 1 Apr 2020 17:39:41 -0400

In article <4C3D0687FFB29333C9ED2F64@PSB> you write:
>job description. ...

Hi from not-Heather.

While Heather definitely knows a lot more than I do about publications
and libraries and archives, I'm finding issues about XML and tools
where it's useful to have a technical understanding of what the tools
do and how they work.

We need to think about the long term implications for an archival
series of having our own semi-custom document format and custom tools,
which needs understanding of both what it means to be archival, and
the inevitable issues of software bitrot.

So while I certainly agree that the position needs specific skills that
typical IETFers don't have, let's not try and pin down details yet.

R's,
John

---

From: John C Klensin <[email-masked]>
Date: Wed, 01 Apr 2020 18:23:30 -0400

--On Tuesday, March 31, 2020 22:02 +0100 Stephen Farrell
<[email-masked]> wrote:

> 
> Hi Eliot,
> 
> I'm sure this'll not get forgotten but just to get it
> on the table...
> 
> In earlier discussions there seemed to be relatively
> broad agreement that the idea that the RSE should be
> responsible for the day-to-day performance of the RPC
> was outdated, esp. now that we have the LLC.
> 
> So I think revising that part of the RSE role description
> is something that should be part of this.

Just a comment (the one promised in my note to Jari hours ago --
long day) for the same table.

The path we followed in moving from one (or 1 3/4) RFC Editor(s)
and staff to separate RPC and RSE functions may turn out to be
important here.  Some of that is in RFC 8700, particularly
Section 3.5.  The rest can probably wait for a later date or is
not important.

The thing that is important is that a traditional (since the
split) part of the RSE's role is to make decisions --often in
conjunction with decisions with RPC staff-- that can affect most
measurements of RPC day-to-day performance.  Most of these are
small but, especially as we go through the transition process to
the new format, others involve quite fundamental decisions about
the tradeoffs about Series quality and getting things right for
the long term versus document throughput.

Having the RPC report and be accountable directly to the IETF
Admin LLC Exec Dir while the RSE is accountable to or via some
other structure is plausible.  Many of us have "day job"
experience with similar management arrangements (often described
as "matrix management" or in other terms).  Sometimes they work.
But requiring that the RSE negotiate essentially technical
(although publications-technical, not network
engineering-technical) decisions with an Exec Dir who is charged
with managing day-to-day performance, especially if that is done
in quantitative terms, could easily put us into a situation we'd
come to regret and, even sooner, change the profile of
otherwise-qualified people who would be willing to apply for the
RSE position.

So, yes, I think we need to review those relationships.  And, in
the context of my response to Jari, I think we need to nail down
what we want (even if it is "no change" in order to write a job
description.  But let's be careful, especially so because at
least a few of us think that part of what got us into this
situation was assuming that the RSE was just another contractor
who could be evaluated, subjected to competitive bids across a
broad number of qualified bidders, and then selected and
obtained/managed the same way we issue an RFP and contract for a
barrel or two  of nails or even a spool of fiber cable.

best,
   john

---

From: Stephen Farrell <[email-masked]>
Date: Thu, 2 Apr 2020 00:50:56 +0100

Hiya,

On 01/04/2020 23:23, John C Klensin wrote:
> a few of us
> think that part of what got us into this situation was assuming that
> the RSE was just another contractor who could be evaluated,

I reckon there's a real element of truth there and I'm
myself quite happy to be open about my opinion that it
is not a good plan for the RSE to be considered to be
"just another contractor." That may be down to a love of
books or some such irrelevant bias of course - I am not
claiming my opinion here is soundly justifiable;-)

That said, there are some IETF participants for whom I
have a lot of respect, who, I think, do in fact consider
the RSE role as one to be filled by "just another
contractor," or maybe they'd describe it differently,
perhaps by stating that the person in the RSE role is
not guaranteed to be a god-like being or some such.

I hope this process can find a way to deal with such
divergent views, or find a role description with which
we are all equally unhappy, but with which we can all
live;-(

Cheers,
S.

---

From: Larry Masinter <[email-masked]>
Date: Wed, 1 Apr 2020 17:38:20 -0700

> I hope this process can find a way to deal with such divergent views, or find
> a role description with which we are all equally unhappy, but with which we
> can all live;-(

I think the question is how big a scope of "solutions" should be taken on.
At the "large scope" are changes to the nature of RFCs and the series
At the "small scope" end I'd suggest -- without any other change -- focusing on the "AUTH48" status since 48 hours and only the authors cannot proofread everything that needs proofreading.

(History RFC2616 had several errors that were due to bugs in the process of converting Word to text/plain)

I can't tell from the posts and intro material what level of changes to the process you'd want to contemplate.

---

From: Christian Huitema <[email-masked]>
Date: Wed, 1 Apr 2020 18:39:53 -0700

> I can't tell from the posts and intro material what level of changes to the process you'd want to contemplate.

There is indeed an issue with AUTH48, but there is a more general issue
with "changing documents after consensus is declared".

For the IETF stream, the current process puts the final phase of editing
after the final vote by the IESG. This is a source of both tensions and
delays. The tension is obvious: if a document can be changed after it
has been approved, how do we guarantee that the changes do not change
the meaning of the documents in a way that alters consensus? The process
puts that responsibility on the authors and the AD, but the authors may
or may not have the time to examine the changes closely, and the AD may
or may not be already overloaded. So that create delays, not because the
copy editors are slow, but because the authors and AD end up responding
slowly.

Removing tensions and delays would make life easier for everybody.

-- Christian Huitema

---

From: "Joel M. Halpern" <[email-masked]>
Date: Wed, 1 Apr 2020 22:22:27 -0400

Christian, I am a bit confused by your message.  Removing post-approval 
editing, while it would reduce tension and delay, would also 
significantly reduce document quality.  The RPC does an impressive job 
of fixing the many minor but significant errors we put in our documents. 
  Ranging from inconsistent terminology to really bad grammar.
I, for one, would be really unhappy at trying to decide we did not need 
that.

Yours,
Joel

---

From: Christian Huitema <[email-masked]>
Date: Wed, 1 Apr 2020 19:30:09 -0700

I did not write that we should remove copy editing. But we could certainly find models in which it happens before approval, not after it.

-- Christian Huitema 

> On Apr 1, 2020, at 7:22 PM, Joel M. Halpern <[email-masked]> wrote:
> 
> Christian, I am a bit confused by your message.  Removing post-approval editing, while it would reduce tension and delay, would also significantly reduce document quality.  The RPC does an impressive job of fixing the many minor but significant errors we put in our documents.  Ranging from inconsistent terminology to really bad grammar.
> I, for one, would be really unhappy at trying to decide we did not need that.
> 
> Yours,
> Joel
> 
>> There is indeed an issue with AUTH48, but there is a more general issue with "changing documents after consensus is declared".
>> For the IETF stream, the current process puts the final phase of editing after the final vote by the IESG. This is a source of both tensions and delays. The tension is obvious: if a document can be changed after it has been approved, how do we guarantee that the changes do not change the meaning of the documents in a way that alters consensus? The process puts that responsibility on the authors and the AD, but the authors may or may not have the time to examine the changes closely, and the AD may or may not be already overloaded. So that create delays, not because the copy editors are slow, but because the authors and AD end up responding slowly.
> -- 
> Rfced-future mailing list
> [email-masked]
> https://www.iab.org/mailman/listinfo/rfced-future

---

From: Michael StJohns <[email-masked]>
Date: Wed, 1 Apr 2020 22:30:31 -0400

> I can't tell from the posts and intro material what level of changes to the process you'd want to contemplate.
I'm pretty sure that is not really part of this.  It shows all the signs 
of being a rat hole.   Maybe write up a proposal entitled "AUTH48 
considered harmful" and propose a fix?

For this set of tasks, let's stick more to figuring out the model of the 
RFC editor (e.g. a range of micro managed peon to completely independent 
entity co-equal with the various I* on one axis and someone who knows 
how to write a complete sentence to someone who has the mad skills to 
develop, manage, edit and publish a world class document series on the 
other).

And before someone asks "what about oversight?", leave it on the table 
for now - that comes easier once you have the model.

Maybe a way of going about this is to look at Jon, Bob/Joyce and Heather 
and identify what each brought to the series and, of those items, what 
we want to see in a future RSE.    Then compare that against what the 
IAB and IESG and other various folk want as changes.  And by changes, I 
mean to the model - changes to the streams or publication policies or 
the use of the RFC name I would suggest strongly are out of scope.

Mike

---

From: Michael StJohns <[email-masked]>
Date: Wed, 1 Apr 2020 22:31:58 -0400

As I suggested in my response to Larry, perhaps this is a rat hole that 
should be avoided at this point in the discussion?

Mike

---

From: Brian E Carpenter <[email-masked]>
Date: Thu, 2 Apr 2020 15:44:58 +1300

> I, for one, would be really unhappy at trying to decide we did not need 

Fully agreed. The WG Chairs and Area ADs are CCed on AUTH48 and I assume they will look out for changes of semantics. If not, why are they copied?

Certainly any substantive changes must be properly reviewed, in the worst case by going back to WGLC. But that is vanishingly rare, I think.

    Brian

---

From: Larry Masinter <[email-masked]>
Date: Wed, 1 Apr 2020 20:14:03 -0700

I think what I'm calling the "AUTH48" problem was greatly exacerbated by the format switch, making the normal publishing process of reviewing "galley proofs" to be too onerous for those familiar with the material vs. the RFC Editor who has enough to worry about without knowing what might or might not be significant elements of typography, spacing, emphasis in formats that the author didn't even review (text, html, pdf)

IMHO, the "real problem" from a publishing workflow point of view  is that there is a workflow task that is not explicitly assigned, namely to validate that the to-be-published RFC, in all its published formats, reflects the intent of the author and the body that approved it.

It's easy to imagine the results would be a lot of unmet expectation and finger pointing, and arguments about what the "real problem" was.
--
https://LarryMasinter.net

---

From: John C Klensin <[email-masked]>
Date: Wed, 01 Apr 2020 23:26:37 -0400

--On Wednesday, April 1, 2020 18:39 -0700 Christian Huitema
<[email-masked]> wrote:

>...

> There is indeed an issue with AUTH48, but there is a more
> general issue with "changing documents after consensus is
> declared".
> 
> For the IETF stream, the current process puts the final phase
> of editing after the final vote by the IESG. This is a source
> of both tensions and delays. The tension is obvious: if a
> document can be changed after it has been approved, how do we
> guarantee that the changes do not change the meaning of the
> documents in a way that alters consensus? The process puts
> that responsibility on the authors and the AD, but the authors
> may or may not have the time to examine the changes closely,
> and the AD may or may not be already overloaded. So that
> create delays, not because the copy editors are slow, but
> because the authors and AD end up responding slowly.
> 

I completely agree with the comments above and would add that,
sometimes, between when IETF Last Call ends and when the IESG
votes, the IESG insists on changes to the document.  In my
experience, the vast majority of those changes improve things
but some come very close to changing substantive details of the
specification.  While the author(s), shepherd, and WG Chair(s)
are normally aware of the changes, returning the revised
document to the WG and through IETF Last Call for a final check
in the substantive issues (possibly a matter of judgment) is
very rare.  The net result is similar to (and, from the outside,
often indistinguishable from) part of the AUTH48 problem you
mention above: the document that is published may differ in
significant (not just minor editorial) ways from the document on
which IETF consensus is reached during Last Call or earlier.

That said, I strongly recommend that we not take that on as part
of this effort.  It is a problem exclusive to the IETF Stream
(other streams may or may not have similar problems).  It
involves significant issues in how documents in that stream are
handled and with the dynamics between the RPC and the IESG (or
particular ADs or IETF stream authors).   It does not
significantly involve the RSE (although I understand that some
RSEs, including Heather, sometimes found themselves trying to
mediate among the various parties).

So, if we want to focus on the issues needed to write a good job
description and get a new RSE on board, this is probably not one
of those issues.  

best,
   john

---

From: "[email-masked]" <[email-masked]>
Date: Wed, 01 Apr 2020 23:43:17 -0400

Given the amount of change I have seen (and I have not been on the IESG in MANY years) during approval,  it would be really hard to do the RPC work before IESG review.  We could do another round of review after RPC.  But that would increase delay, not decrease it.Yours,JoelSent via the Samsung Galaxy S7, an AT&T 4G LTE smartphone
-------- Original message --------From: Christian Huitema <[email-masked]> Date: 4/1/20  22:30  (GMT-05:00) To: "Joel M. Halpern" <[email-masked]> Cc: [email-masked] Subject: Re: [Rfced-future] Welcome to the RFC Editor Future Development Program I did not write that we should remove copy editing. But we could certainly find models in which it happens before approval, not after it.-- Christian Huitema > On Apr 1, 2020, at 7:22 PM, Joel M. Halpern <[email-masked]> wrote:> > Christian, I am a bit confused by your message.  Removing post-approval editing, while it would reduce tension and delay, would also significantly reduce document quality.  The RPC does an impressive job of fixing the many minor but significant errors we put in our documents.  Ranging from inconsistent terminology to really bad grammar.> I, for one, would be really unhappy at trying to decide we did not need that.> > Yours,> Joel> >> On 4/1/2020 9:39 PM, Christian Huitema wrote:>> On 4/1/2020 5:38 PM, Larry Masinter wrote:>>>> I hope this process can find a way to deal with such divergent views, or find>>>> a role description with which we are all equally unhappy, but with which we>>>> can all live;-(>>> I think the question is how big a scope of "solutions" should be taken on.>>> At the "large scope" are changes to the nature of RFCs and the series>>> At the "small scope" end I'd suggest -- without any other change -- focusing on the "AUTH48" status since 48 hours and only the authors cannot proofread everything that needs proofreading.>>> >>> (History RFC2616 had several errors that were due to bugs in the process of converting Word to text/plain)>>> >>> I can't tell from the posts and intro material what level of changes to the process you'd want to contemplate.>> There is indeed an issue with AUTH48, but there is a more general issue with "changing documents after consensus is declared".>> For the IETF stream, the current process puts the final phase of editing after the final vote by the IESG. This is a source of both tensions and delays. The tension is obvious: if a document can be changed after it has been approved, how do we guarantee that the changes do not change the meaning of the documents in a way that alters consensus? The process puts that responsibility on the authors and the AD, but the authors may or may not have the time to examine the changes closely, and the AD may or may not be already overloaded. So that create delays, not because the copy editors are slow, but because the authors and AD end up responding slowly.>> Removing tensions and delays would make life easier for everybody.>> -- Christian Huitema> > -- > Rfced-future mailing list> [email-masked]> https://www.iab.org/mailman/listinfo/rfced-future

---

From: John C Klensin <[email-masked]>
Date: Wed, 01 Apr 2020 23:51:18 -0400

(I've changed the subject line because I really hope we can
split this off and either set it aside until other work is
finished or find another way to handle it as an issue.)

--On Wednesday, April 1, 2020 19:30 -0700 Christian Huitema
<[email-masked]> wrote:

> I did not write that we should remove copy editing. But we
> could certainly find models in which it happens before
> approval, not after it.

Christian, see my earlier note and Mike's response, with while I
agree.  

However, fwiw, models are well-known in which documents reach a
certain state before the final community decision process that
would be nearly equivalent (as process steps) to the combination
of our Last Call and IESG final review are pulled out and
subjected to what is essentially final editing.  There is then
usually an opportunity for the group that produced the draft
document to review it and argue with the editors.  The edited,
document then goes into final reviews.  If those reviews
identify changes that require changes, the changes are made and
the document re-edited, and then, depending on the extent of the
changes, either the whole document or the changes go through the
review process again.

A few details and small differences aside, that is the method
used in ITU-T, in ISO and ISO/IEC JTC1, in many (perhaps most)
ISO National Member Bodies, and in a variety of other SDOs.  The
IETF is probably not unique in the way we do things, but I
suspect we don't have a lot of company.

I'm aware of a number of proposals to shift to something closer
to the other model, at least on during Heather's term and the
first probably in the late 1980s.  The decision to not do so was
always based on what were considered bad tradeoffs with multiple
issue, almost always including:

(1) The editing process takes time. The IETF has never been
willing to have a WG finish a document and then go into a wait
state for some weeks or months while that document is being
edited, then wake up, review it, and recommend that it be put
out for Last Call.   At least in the past, our way of working
was perceived as incompatible with that.

(2) Editing documents that may never be approved, or that will
be approved only with significant changes (and then edited
again), is costly in people and hours.  I vaguely recall Heather
(probably with help from Sandy and/or Alice) making some rough
estimates of how much additional staff would be needed.  I don't
remember the numbers, but it was certainly enough to affect the
bottom line.  When I remember the idea being reviewed, those
potential costs were a consideration.

I do think that things have changed enough that it is time to
ask the question again.  But, again, I think this effort is not
the right place or time.

  best,
    john

---

From: Larry Masinter <[email-masked]>
Date: Wed, 1 Apr 2020 22:23:42 -0700

> As I suggested in my response to Larry, perhaps this is a rat hole that should
> be avoided at this point in the discussion?

That's fine. I just meant AUTH48 as an example of a "small" problem.
Feel free to propose a smaller (non-)rathole that would make the
Editor job less onerous and the qualifications more fillable.

> >>>> At the "large scope" are changes to the nature of RFCs and the
> >>>> series At the "small scope" end I'd suggest -- without any other change -- 
> >>>> focusing on the "AUTH48" status since 48 hours and only the authors 
> >>>>cannot proofread everything that needs proofreading.
> >>>>
> >>>> (History RFC2616 had several errors that were due to bugs in the
> >>>> process of converting Word to text/plain)
> >>>>
--
https://LarryMasinter.net 
https://going-remote.info 

About rats:
https://www.nytimes.com/2020/04/01/science/coronavirus-animals-wildlife-goats.html

---

From: Russ Housley <[email-masked]>
Date: Thu, 2 Apr 2020 11:52:14 -0400

> On Apr 1, 2020, at 11:51 PM, John C Klensin <[email-masked]> wrote:
> 

The IESG approval process almost always leads to changes based in DISCUSS or COMMENT.  Sometimes these are technical changes.  We should not do copy edits until the technical document is stable.  That is exactly where the copy editing is in the current process.

We did a few process experiments where the RFC Editor was invited to do copy edits earlier in the process.  Theses experiments did not reduce the review, approval, or final copy editing times significantly.  My conclusion is that the copy editing is in the right place in the process.

The use of the diff files has made the copy edit review MUCH easier that it used to be.  I'd be very happy to explore what other tools could make it even easier.

Russ

---

From: Ted Lemon <[email-masked]>
Date: Thu, 2 Apr 2020 12:12:38 -0400

> The IESG approval process almost always leads to changes based in DISCUSS or COMMENT.  Sometimes these are technical changes.  We should not do copy edits until the technical document is stable.  T…

One thing to bear in mind is that it is not uncommon to discover a technical issue in AUTH48, just as we do with errata.  I don’t think the right thing to do is to say “don’t fix it, since it’s AUTH48.”  I also don’t think the right thing to do is to say “just fix it and don’t check with the WG.”  However, in at least one case, we had some serious problems that were caught in AUTH48 after the WG had disbanded.  We fixed these, and I think did an IETF last call, although this was ten years ago and my recollection is imperfect.

I say this not to propose a solution, but simply to point out that this is a nasty problem.  I don’t think there’s an easy solution, but I’m sure that we could define a formal process if there were energy to do this.  I agree with other commenters who say that this is out of scope for the current effort, since this is really an IETF-track problem, not an all-track problem.

---

From: Russ Housley <[email-masked]>
Date: Thu, 2 Apr 2020 12:51:49 -0400

> On Apr 2, 2020, at 12:12 PM, Ted Lemon <[email-masked]> wrote:
> 
> On Apr 2, 2020, at 11:52 AM, Russ Housley <[email-masked] <mailto:[email-masked]>> wrote:

This is very rare, but it does happen.  When it does, the AD needs to decide if the fix is obvious or if the fix need to go back to a WG or IETF Last Call.

Russ

---

From: Ted Lemon <[email-masked]>
Date: Thu, 2 Apr 2020 12:55:21 -0400

> This is very rare, but it does happen.  When it does, the AD needs to decide if the fix is obvious or if the fix need to go back to a WG or IETF Last Call.

I agree.  Just saying, there could be a more detailed policy written if someone wants it, and there is no way to make this problem go away that doesn’t guarantee worse outcomes.

---

From: Carsten Bormann <[email-masked]>
Date: Thu, 2 Apr 2020 19:12:21 +0200

> I agree.  Just saying, there could be a more detailed policy written if someone wants it, and there is no way to make this problem go away that doesn’t guarantee worse outcomes.

Right.  WG members that pay attention can already observe the progression of the /authors/ drafts, but it would be nice if the WG was involved more early if changes that are not purely editorial (whatever that means) are being made.  Usually, the WG can just watch and silently agree, so there wouldn’t be additional overhead.

But let me agree here that we are doing this in the right order.  It is fundamentally impossible to avoid minor reiterations at the last process step, and my experience has been that this has mostly been handled well, with a few surprises that happened anyway.
(There is also the RFC 7386/RFC 7396 thing that happened, and might not have if more people were invited to pay attention during AUTH48.  But then RFC 7158/7159 demonstrated that accidents can happen even in the post-AUTH48 processing steps.)

Grüße, Carsten

PS.: And while I was writing this, in a WG Jabber:
…"Please let me know when it goes to Auth48 so I can re-check the examples.”…
Another reason to involve the WG in AUTH48.

---

From: Michael StJohns <[email-masked]>
Date: Thu, 2 Apr 2020 13:16:21 -0400

Trying again to move this to the appropriate list.  Mike

---

From: Eliot Lear <[email-masked]>
Date: Thu, 2 Apr 2020 19:25:58 +0200

All:

Can we distill the high level point that needs addressing in this process?

Are the touch points and flow the responsibility of the RFC Editor to define, or should that be this group?

Eliot

---

From: Ted Lemon <[email-masked]>
Date: Thu, 2 Apr 2020 13:39:20 -0400

> Are the touch points and flow the responsibility of the RFC Editor to define, or should that be this group?

My conclusion from this discussion is that the AUTH48 question is out of scope for this group.

---

From: John C Klensin <[email-masked]>
Date: Thu, 02 Apr 2020 15:31:50 -0400

--On Thursday, April 2, 2020 11:52 -0400 Russ Housley
<[email-masked]> wrote:

> The IESG approval process almost always leads to changes based
> in DISCUSS or COMMENT.  Sometimes these are technical changes.
> We should not do copy edits until the technical document is
> stable.  That is exactly where the copy editing is in the
> current process.
> 
> We did a few process experiments where the RFC Editor was
> invited to do copy edits earlier in the process.  Theses
> experiments did not reduce the review, approval, or final copy
> editing times significantly.  My conclusion is that the copy
> editing is in the right place in the process.
> 
> The use of the diff files has made the copy edit review MUCH
> easier that it used to be.  I'd be very happy to explore what
> other tools could make it even easier.

Russ,

I was just trying to explain that the alternative Christian
seemed to have been suggesting had been used (and was in use) in
other places and how it worked.  I was not, and would not, argue
for it for the IETF because making it work here would, I
believe, require major changes in procedures and maybe even in
culture.

An observation just in case it turns out to be useful in other
contexts.  What the Production Center does often goes beyond
what most of the publishing industry considers copy editing.  It
rarely gets far into the range of what is often described as
technical editing (or, more accurately, technical rewriting),
but it, especially when combined with changes made at the behest
of the IESG after Last Call, goes into that territory.  The
boundary, more or less, is when subject matter expertise, even
expertise on the vocabulary and basic concepts of the relevant
field and/or fairly extensive familiarity with other works in
the same area(s) is required to do the expected editing job.

The distinction is important for two reasons.  First, if all
they did was simple copy editing, we either wouldn't need AUTH48
or AUTH48 would be confined to things like misspellings they had
missed.   They do much more, they have acquired a great deal of
expertise in our documents along the line, their work is
important to us, and describing what they do as copy-editing not
only fails to recognize that but risks our slipping into the
problem of treating the RPC and its staff as just another set of
basically interchangeable contractors.

Second, my concern (and, if I understood Christian's note, at
least part of his) is that we have too many documents that get
to the end of community review during Last Call and then change
enough via IESG action or RPC efforts to be only barely
recognizable as the same document.  The changes may be
substantive ones that the IESG considers minor or the writing
styles or skills in writing technical English of some authors
may have required major rewrites of paragraphs or sections by
the RPC.  I've rarely seen either cause harm, but the fact
remains that the community is signing off on one document and
then a different document is being published.  At least in
principle, that is a risk and maybe se should be thinking about
it and whether adjustments are in order (with the understanding
that reordering the editing and approval processes is not the
only option).

best,
   john

---

From: "John Levine" <[email-masked]>
Date: 2 Apr 2020 15:58:30 -0400

>Are the touch points and flow the responsibility of the RFC Editor to define, or should that be this group?

It is a fact of life that errors will always turn up at inconvenient
points in the process.  One time I found a mistake in one of my books
while doing the equivalent of AUTH48.  I found that the same error was
in two previous editions, and none of my coauthor, our quite competent
editors, or me had noticed it before.

We need to consider the tradeoffs between fixing them and living with
them, with the costs of fixing them being delay and perhaps rework,
and the possibility that the fix will be wrong, too.

R's,
John

---

From: "John Levine" <[email-masked]>
Date: 2 Apr 2020 15:59:47 -0400

In article <[email-masked]> you write:
>IMHO, the "real problem" from a publishing workflow point of view  is that there is a workflow task that is not explicitly
>assigned, namely to validate that the to-be-published RFC, in all its published formats, reflects the intent of the author
>and the body that approved it.

Trust me, that's not possible.

R's,
John

---

From: "Andrew G. Malis" <[email-masked]>
Date: Thu, 2 Apr 2020 17:01:08 -0400

John,

I'd like to dig into your reply a bit. We've both been through our share of
AUTH48s. When I get the proposed changes and questions back from the RFC
Editor, I do my best to make sure that the primary output format (until
recently, the text format, and going forward, the HTML format) will be
correct from a content point of view and hasn't changed
anything technically substantive - changes are strictly editorial or to
correct errors like typos. If necessary, I've consulted with the
responsible AD to make sure that she or he agrees with the proposed
changes. I then trust the RFC Editor that the secondary formats (PDF, etc.)
will match the primary from a content viewpoint.  But following this
process, I'm fairly certain that "the to-be-published RFC, in all its
published formats, reflects the intent of the author and the body that
approved it."  So I'm curious why you feel that's not possible.

Cheers,
Andy

On Thu, Apr 2, 2020 at 3:59 PM John Levine <[email-masked]> wrote:

> >IMHO, the "real problem" from a publishing workflow point of view  is
> that there is a workflow task that is not explicitly
> >assigned, namely to validate that the to-be-published RFC, in all its
> published formats, reflects the intent of the author

---

From: "John R. Levine" <[email-masked]>
Date: 2 Apr 2020 17:09:39 -0400

> But following this process, I'm fairly certain that "the to-be-published 
> RFC, in all its published formats, reflects the intent of the author and 
> the body that approved it."  So I'm curious why you feel that's not 
> possible.

Usually it will, but sometimes no matter how careful everyone is, 
sometimes they'll miss something.  I'm certainly not saying that we 
shouldn't try to make the quality as good as possible, but I am saying 
that as good as possible is not perfect and our processes have to be able 
to deal with that reality.

Sometimes it slips through all the way to the published RFC, which is why 
we have errata.  We have a lot of them, not infrequently for documents 
published many years ago.

Regards,
John Levine, [email-masked], Primary Perpetrator of "The Internet for Dummies",
Please consider the environment before reading this e-mail. https://jl.ly

---

From: Brian E Carpenter <[email-masked]>
Date: Fri, 3 Apr 2020 10:46:18 +1300

On 03-Apr-20 06:25, Eliot Lear wrote:
> All:
> 
> Can we distill the high level point that needs addressing in *this* process?
> 

Definitely the former. If we talk about such things here, we are already micro-managing at least as badly as the RSOC seems to have done. Actually, I'm simply deleting the thread at this point, so it's lucky I saw Eliot's question.

   Brian

---

From: Russ Housley <[email-masked]>
Date: Thu, 2 Apr 2020 20:11:14 -0400

John:

> that reordering the editing and approval processes is not the

In the IETF, WG chairs, review teams, and ADs all have a role in turning back documents that are so poorly written that they need more that just copy editing.  I know that I have tried to do this in my working groups.  It is hard to tell someone that their document needs mush more work to be ready to progress, but that is the right place to apply the judgement.

The RFC Editor ought to be able to tell the approving stream that the document is not ready for copy editing as well.  I understand that the RFC Production Center is very reluctant to do that.

Russ

---

From: Christian Huitema <[email-masked]>
Date: Thu, 2 Apr 2020 17:30:43 -0700

> The RFC Editor ought to be able to tell the approving stream that the document is not ready for copy editing as well.  I understand that the RFC Production Center is very reluctant to do that.

How is "your document needs mush more work to be ready to progress" any
different from "I found five attacks that your security considerations
do not address" ?

-- Christian Huitema

---

From: Larry Masinter <[email-masked]>
Date: Thu, 2 Apr 2020 20:01:18 -0700

> I then trust the RFC Editor that the secondary formats (PDF, etc.) will match the primary from a content viewpoint.  

There’s a reason why typical publishing workflows involve proofreading a “galley proof”

the already “typeset” format with the exact placement.

I don’t want to belabor the “AUTH48” problem except as an example of the mismatch of expectation (trust) against something not in the job description might lead to problems.

During the AUTH48 period, whoever is reviewing the document needs to check ALL of the formats, especially around examples and figures and any non-ascii text.

---

From: "John R. Levine" <[email-masked]>
Date: 2 Apr 2020 23:07:51 -0400

> During the AUTH48 period, whoever is reviewing the document needs to check ALL of the formats, especially around examples and figures and any non-ascii text.

Currently the RPC people are proofreading all of the output formats, which 
takes a lot of work.  I hope authors are also, but that's up to them.

Keeping in mind that only the XML is canonical, and that PDF layout turns 
out to be surprisingly hard, one of the things we're considering to keep 
the documents flowing is to live with less than perfect PDF rendering for 
now, and to rerender them later once we understand the issues better.

I am NOT suggesting that we thrash this out now, just that we need to keep 
that not every file one might see is necessarily part of the permanent 
archive.

Regards,

---

From: Julian Reschke <[email-masked]>
Date: Fri, 3 Apr 2020 06:46:31 +0200

On 03.04.2020 05:07, John R. Levine wrote:
>> During the AUTH48 period, whoever is reviewing the document needs to
>> check ALL of the formats, especially around examples and figures and
>> any non-ascii text.
>
> Currently the RPC people are proofreading all of the output formats,
> which takes a lot of work.  I hope authors are also, but that's up to them.
> ...

That sounds like a bug in the process to me. And it won't scale if we
add ePub.

There should be *one* format that is to be proofread, HTML. If the other
output formats have an issue, that's a tooling problem that needs to be
solved, and the other formats can be regenerated.

Best regards, Julian

---

From: Carsten Bormann <[email-masked]>
Date: Fri, 3 Apr 2020 07:25:26 +0200

> solved, and the other formats can be regenerated.

This doesn’t work too well in an environment where the tooling is fluid, and in fact the format that the tooling works from and its interpretation is fluid as well.

When the tooling has a bug (read: it uses the interpretation of 2020-04-02), and the XML is adapted to work around this bug, and the tooling is then fixed (read: it now uses the interpretation of 2020-04-03), the new output may be erroneous.

Putting the “to be interpreted with the bugs of 2020-04-02” into the XML makes this workable, but it might be preferable to give up the idea that the XML is immutable and allow up-conversion into the format of 2020-04-03.

We probably should have recognized that we would be working with a beta v3 for a while.  We could still recognize that there will be a v4, and a v5, ...

I’m not saying that I have a solution to this problem.  Proofreading the HTML is probably the right thing for an author (in particular if not an XML aficionado).  One view at the TXT is prudent as well.

Grüße, Carsten

---

From: "Joel M. Halpern" <[email-masked]>
Date: Fri, 3 Apr 2020 01:36:44 -0400

Julian, on what basis do you assert that the HTML is all that matters. 
As far as I know, the agreements we have in place make the HTML one of 
the output formats, no more and no less privileged. You may want a 
different answer, but until we have rough consensus on such a different 
answer, asserting one does not make it so.

Yours,
Joel

---

From: Julian Reschke <[email-masked]>
Date: Fri, 3 Apr 2020 08:59:52 +0200

> This doesn’t work too well in an environment where the tooling is fluid, and in fact the format that the tooling works from and its interpretation is fluid as well.

If the tooling is that fluid (is it for HTML?), that shows that we
shouldn't have started the switch yet...

> When the tooling has a bug (read: it uses the interpretation of 2020-04-02), and the XML is adapted to work around this bug, and the tooling is then fixed (read: it now uses the interpretation of 2…

The (canonical, published) XML never ever should be adjusted to work
around a bug. Is this happening?

Best regards, Julian

---

From: Julian Reschke <[email-masked]>
Date: Fri, 3 Apr 2020 09:02:29 +0200

> answer, asserting one does not make it so.

It's a practical observation.

- TXT can not support all features of the vocabulary, thus just checking
the TXT in general would not be sufficient

- PDF generation (IMHO) has had less development time, and is (IMHO) is
less used

Best regards, Julian

---

From: "John R. Levine" <[email-masked]>
Date: 3 Apr 2020 11:48:43 -0400

> solved, and the other formats can be regenerated.

Yeah, that's why I've been suggesting we temporarily live with less than 
beautiful PDF output.

Regards,

---

From: John C Klensin <[email-masked]>
Date: Fri, 03 Apr 2020 15:41:13 -0400

--On Thursday, April 2, 2020 19:25 +0200 Eliot Lear
<[email-masked]> wrote:

> Are the touch points and flow the responsibility of the RFC
> Editor to define, or should that be this group?

Eliot, 

I think "touch points and flow" is an oversimplification and
that this thread illustrates that.  Let me see if I can divide
things up without going on too long about it.

(1) This particular issue is out of scope because the problem is
probably exclusive to the IETF streams.  If other streams have
the same issues, we haven't heard about them, but, even if they
have, per-stream solutions are likely to be needed.   I don't
think there is any reason to make reconsideration of the stream
model part of this process.

(2) At the same time, the question of the responsibilities and
authority of the RFC Editor Function seems to me to be very much
on-topic.  Historically, the RFC Editor has always had the
authority to say "this is not publishable in its current form,
please fix it" (see Appendix A.2 of draft-flanagan-7322bis-05,
posted today).  In ancient times (i.e., during the
Postel-Reynolds period) drafts were periodically bounced because
they were not coherent enough to be editable with reasonable
resources.  Btw, in those times, drafts were also bounced
because the RFC Editor judged them to be technically incoherent.
And, sometimes the RFC Editor would apply a "fix" to a document
that had substantive technical input... and a few incidents of
that were what set us on the path to having an AUTH48 process.
However, in more recent times, when the RFC Editor has pushed
back (and I'm not aware of that occurring in many years, so what
I'm about to say is not a criticism of any present or recent
IESG), the responses from the IESG were rather hostile.. and
that is why we have an explicit provision that says the IESG can
tell the RSE/RPC to publish a document exactly as the IESG
prefers with the RFC Editor able to only add a note indicating
that occurred.

I think that immediately goes to two types of questions I think
are in scope and, indeed, critical for this group:

(a) Noting that the decision to edit (or not) a problematic
document will often have consequences on overall workflow, do we
want the decision as to whether to do so be made by the RPC in
conjunction with (or under the authority of) the RSE or under
the authority of the Executive Director (who may have no
publication or editing experience).  If general guidelines are
developed within the RFC Editor Function about the properties
that should cause a document to be pushed back, is that an RSE
function, an RPC one without the RSE, of an Executive Director
matter?

(b) Is the RSE ultimately independent or simply a contractor who
can be managed, and whose decisions can be second-guessed, by
the IESG, IAB, RSOC, or some other group, either directly or via
the Executive Director?

thanks,
   john

---

From: Russ Housley <[email-masked]>
Date: Fri, 3 Apr 2020 16:06:05 -0400

> On Apr 2, 2020, at 8:30 PM, Christian Huitema <[email-masked]> wrote:
> How is "your document needs mush more work to be ready to progress" any different from "I found five attacks that your security considerations do not address" ?
> 

The first is a statement about the clarity of the writing in the document.  The second is a statement about the technical quality of the document.  I do not expect the RFC Editor to make the second assessment.

Russ

---

From: Brian E Carpenter <[email-masked]>
Date: Sat, 4 Apr 2020 09:32:38 +1300

Julian,
> shouldn't have started the switch yet...

But we did, so that ship has sailed.
 
> around a bug. Is this happening?
 
<heresyAlert>It should be happening. For the period during which v3 is in flux, we should indeed update already-published v3 XML files, not constantly, but on some sort of snapshot or checkpoint basis. Obviously we must not update the substantive content, and we should have an audit trail, but only by doing such updates can we have a reasonable situation at the end.

If there isn't a v3.n sub-version number in the XML files today, there certainly should be.</heresyAlert>

    Brian

---

From: Carsten Bormann <[email-masked]>
Date: Fri, 3 Apr 2020 22:36:30 +0200

On 2020-04-03, at 22:32, Brian E Carpenter <[email-masked]> wrote:
> 
> <heresyAlert>It should be happening.

Yes!

> For the period during which v3 is in flux, we should indeed update

s/update/up-convert/

> already-published v3 XML files, not constantly, but on some sort of snapshot or checkpoint basis. 

Grüße, Carsten

---

From: "John R. Levine" <[email-masked]>
Date: 3 Apr 2020 16:46:17 -0400

>> around a bug. Is this happening?

I sure hope not.  The RPC now has a document state TI that means that a 
document is waiting for tools to be fixed.  That's happened a few times 
but so far the fixes have been simple enough that Henrik has been able to 
do them without unduly delaying the documents.

Regards,

---

From: John C Klensin <[email-masked]>
Date: Fri, 03 Apr 2020 23:23:05 -0400

--On Thursday, April 2, 2020 20:11 -0400 Russ Housley
<[email-masked]> wrote:

>> to the end of community review during Last Call and then
>> change enough via IESG action or RPC efforts to be only barely
>> principle, that is a risk and maybe se should be thinking
>> about it and whether adjustments are in order (with the
>> understanding that reordering the editing and approval
>> processes is not the only option).
> 
> In the IETF, WG chairs, review teams, and ADs all have a role
> in turning back documents that are so poorly written that they
> need more that just copy editing.  I know that I have tried to
> do this in my working groups.  It is hard to tell someone that
> their document needs mush more work to be ready to progress,
> but that is the right place to apply the judgement.

Yes.  And that also means being much more careful about saying
"the RFC Editor will sort that out" -- something we heard
Heather comment on or complain about several times.  There are
many things about which that is appropriate and most of them
really are copy editing (capitalization, commas, grammar,...),
but many things that may not be.

> The RFC Editor ought to be able to tell the approving stream
> that the document is not ready for copy editing as well.  I
> understand that the RFC Production Center is very reluctant to
> do that.

Yes.  In part I assume because they have been treated badly when
they have tried to do that in the past (I knew several
instances, long ago, in which that did occur and assume that is
part of what made them reluctant).

I think this is something else we should come back to after we
have a new RSE in place and perhaps when we are far enough into
the xml2rfc v3 transition to have a clearer picture of levels of
effort but I wonder if things could be improved by asking the
RPC to make a very quick, triage-level, pass through a document
at about the time the IESG starts a Last Call.  The intention
would not be to start editing but rather to be able to give the
IESG quick advice -- advice for which the RPC would not be taken
to task if they got wrong -- about how much work and of what
sorts the document would need if it was approved in that form.
I could see the IESG, assisted by that sort of report,
tentatively approving a document after Last Call but returning
it to a WG for sorting out the explanation of technical
editorial details to the point that the intent of the
specification would be clear to all readers.  That wouldn't
eliminate the need for the RPC to do a copy editing job (and a
lightweight technical editing one), but it might well eliminate
many or most of the "did you really intend this to mean X or Y"
discussions at AUTH48 with all of the risks such questions imply.

best,
   john

---

From: Christian Huitema <[email-masked]>
Date: Sat, 4 Apr 2020 08:55:10 -0700

-- Christian Huitema 

> On Apr 3, 2020, at 1:06 PM, Russ Housley <[email-masked]> wrote:
> 

I think people elaborated on that thread, but it boils down to being able to push back on documents that are in a bad shape and would require too much editing work. Of course this will create some push back from authors or from the IESG. I think the reformed process should be explicit about how to handle such push back.

-- Christian Huitema

---

From: John C Klensin <[email-masked]>
Date: Sat, 04 Apr 2020 12:57:23 -0400

--On Saturday, April 4, 2020 09:32 +1300 Brian E Carpenter
<[email-masked]> wrote:

>>>>>> During the AUTH48 period, whoever is reviewing the
>>>>>> document needs to check ALL of the formats, especially
>>>>>> around examples and figures and any non-ascii text.
>>>>> 
>>>>> Currently the RPC people are proofreading all of the
>>>>> output formats, which takes a lot of work.  I hope authors
>>>>> are also, but that's up to them. ...
>>>> 
>>>> That sounds like a bug in the process to me. And it won't
>>>> scale if we add ePub.
>>>> 
>>>> There should be *one* format that is to be proofread, HTML.
>>>> If the other output formats have an issue, that's a tooling
>>>> problem that needs to be solved, and the other formats can
>>>> be regenerated.
>>> 
>>> This doesn't work too well in an environment where the
>>> tooling is fluid, and in fact the format that the tooling
>>> works from and its interpretation is fluid as well.
>> 
>> If the tooling is that fluid (is it for HTML?), that shows
>> that we shouldn't have started the switch yet...
> 
>>> When the tooling has a bug (read: it uses the interpretation
>>> of 2020-04-02), and the XML is adapted to work around this
>>> bug, and the tooling is then fixed (read: it now uses the
>>> interpretation of 2020-04-03), the new output may be
>>> erroneous.
>> 
>> The (canonical, published) XML never ever should be adjusted
>> to work around a bug. Is this happening?
>  
> <heresyAlert>It should be happening. For the period during
> which v3 is in flux, we should indeed update already-published
> v3 XML files, not constantly, but on some sort of snapshot or
> checkpoint basis. Obviously we must not update the substantive
> content, and we should have an audit trail, but only by doing
> such updates can we have a reasonable situation at the end.
> 
> If there isn't a v3.n sub-version number in the XML files
> today, there certainly should be.</heresyAlert>

Agreed.  A bad solution but, given that we made the switch
while, at least retrospectively, in experimental mode, almost
certainly less bad than others.

<probably-worseHeresyAlert>
I know it is widely believed that HTML is so important in the
world and so widely used that it will survive in its current
form for centuries if not millennia, however, it is almost
certainly the least stable of the three output  forms for at
least two reasons.  First, in the circa 30 years since it
appeared, the community has already gone through five versions
and is working on a sixth (or, if you prefer, tuning the 5th).
Even more extremely, normally-sensible people are claiming that
HTML has outlived its usefulness and should be replaced by
something they (or their colleagues) have invented.  AFAICT,
none of those have gotten significant traction for more than
specific applications, but there is certainly no way to
guarantee that state will continue forever, i.e., that nothing
will come along that is actually sufficiently better that it
becomes the norm and HTML a historical legacy and artifact.   It
seems hard to be certain that whatever we do now will be easily
and compatibly rendered by whatever passes for a browser in even
75 or 100 years.  The biggest advantage of using HTML as one of
the standard output formats -- that it can be interpreted and
rendered passably well on a variety of different devices with
appropriate style sheets -- is also a risk if we consider it the
"real" format (e.g., the only one that is subject to final
("AUTH48" or whatever we call it) review, at last as long as we
care at all about presentation. It is a specification format,
not a presentation format and, even as a specification format,
it is not self-contained, depending instead on style sheets and
local conventions to precise specify presentation.   If one is
working with a document, trying to create new versions, etc.,
XML as the official/archival form is probably ok and most of us
might prefer to work with HTML or even plain text.  But, if one
is concerned about archival availability of a presentation
format, it is either the PDF/A for which proofreading and
layout-checking is important or we need to be checking all three
formats to be sure they yield plausible results.
</probably-worseHeresyAlert>

---

From: "John Levine" <[email-masked]>
Date: 4 Apr 2020 13:52:02 -0400

>certainly less bad than others.

My preference would be once we've defined the final XML vocabulary, go
back and adjust the XML (not the text) in the published RFCs to match.
Why force people for the next 50 years to live with our goofs?

The changes won't be very large since the changes since v3 are still
pretty minor.  FWIW all the changes so far are backward compatible
so all published RFCs are valid with the current version.  The changes
to the XML would be if we decide to back out or modify some of the
changs to the spec.

>least two reasons. ...

Actually, I agree.  I am sure there will be something called HTML
for many decades (sort of like there's something called Fortran)
but it won't be all that much like what we have now.  That's why
the HTML is an output format, not canonical.

On the other hand, HTML is expressive enough that if we can get
a good quality translation of the XML into whatever HTML is now,
that should be the main place for proofreading.

R's,
John

---

From: Mark Nottingham <[email-masked]>
Date: Thu, 16 Apr 2020 16:47:24 +1000

Hi Brian,

Sorry for the delay, things intervened.

> On 1 Apr 2020, at 1:17 pm, Brian E Carpenter <[email-masked]> wrote:
> 
>>>> My preference is that we define the job such that it is clearly the RFC Editor's job to lead that discussion / process.
>>> 
>>> I very strongly agree with that, but to my mind that requires that we agree on the principle that the RFC Editor is autonomous and the series is not controlled by the IETF or, except at a very abstract level, by the IAB.
>> 
>> I don't see how that follows. We have plenty of people who lead discussions and processes in this community, and yet they still have oversight, checks and balances. There may be reasons for exempting the RSE from them, but declaring that special status as a presumption seems odd in a program with our scope.
> 
> I won't repeat what I said in https://www.ietf.org/archive/id/draft-carpenter-request-for-comments-01.txt, but I will assume you've read it.

I had, but I've re-read it to refresh my memory. FWIW, I agree with much of what you say there, but don't always share your conclusions. Happy to discuss that when and where it's appropriate.

> Of course, being autonomous shouldn't mean exemption from checks and balances. However, in the case of the RSE (and unlike most of our activities), it really isn't clear which community should implement those checks and balances. And the evidence from the IAOC and the RSOC is that we aren't very good at designing oversight functions. I think that's our biggest challenge.

Agreed regarding the challenge. Perhaps "autonomous" isn't the right word; it just struck me as odd. 

Cheers,

--
Mark Nottingham   https://www.mnot.net/