Subject: [Ietf-and-github] Mirja Kühlewind's No Objection on draft-ietf-git-using-github-05: (with COMMENT)

From: Mirja Kühlewind via Datatracker <[email-masked]>
Date: Tue, 10 Mar 2020 05:46:30 -0700

Mirja Kühlewind has entered the following ballot position for
draft-ietf-git-using-github-05: No Objection

When responding, please keep the subject line intact and reply to all
email addresses included in the To and CC lines. (Feel free to cut this
introductory paragraph, however.)

Please refer to https://www.ietf.org/iesg/statement/discuss-criteria.html
for more information about IESG DISCUSS and COMMENT positions.

The document, along with other ballot positions, can be found here:
https://datatracker.ietf.org/doc/draft-ietf-git-using-github/

----------------------------------------------------------------------
COMMENT:
----------------------------------------------------------------------

Thanks for this well-written and helpful document. I have a couple of issue
below that touch on the use of normative language and relation to RFC2418 (as
well as maybe RFC2026). I guess those could qualify as discuss but I leave them
as comments for now trusting in the editors and AD to address them
appropriately.

One general comment:
I don't really understand the relationship between this document and
draft-ietf-git-github-wg-configuration. There is a lot of direct overlap
especially in section 2 but also in other sections. I understand that this
document is BCP and such more appropriate to specify practices normatively but
I don't really see why draft-ietf-git-github-wg-configuration is still needed
then. The only part that is left to draft-ietf-git-github-wg-configuration are
ideas about how things could be integrated in the datatracker and with the
secretariat. As I say in my discuss ballot for that draft, it is not clear to
my why we need an RFC for that at this point of time. If no decision is made
yet, this could have also just be integrated e.g. into the appendix on this
draft (or even partially in the security considerations section).

Other comments/questions:

1) Sec 3.2
"...or they might create repositories for individual documents on request."
I made a similar comment in my ballot for
draft-ietf-git-github-wg-configuration. I think creating repos for individuals
document is maybe not always the best option. I assume the intention here to
not out-rule any cases where this might be okay (e.g. if a document is expectd
to be adopted soon), however, I would rather prefer to see a recommendation to
usually not do that as there can be many individual drafts in the groups and it
can provide a wrong signal if repos are created for some and not others.

2) Sec 4.1.3:
"   Issues that have reached a resolution that has Working Group
   consensus MUST NOT be reopened unless new information is presented."
I do understand the intention here as it is important to make process, however,
inline with the rest of the document and the general idea that practises can
vary from group to group, I think this really should be a SHOULD NOT only.

Also further
"Resolved issues MUST remain
   closed unless there is consensus to reopen an issue."
I find this actually a bit strange because it should also then say something
like "Resolved issues MUST only be closed with wg consensus"... I guess the
word "resolved" is important here but why would you re-open an issue at all if
it is resolved? I guess the case is that the wg figures that the issue is not
resolved yet. Is the recommendation here for a wg participant to open a new
issue if he/she thinks an issue is not fully resolved yet (rather than just
re-open)? If so, you should say that.

3) Sec 4.2:
"   Editors SHOULD make pull requests for all substantial changes rather
   than committing directly to the "master" branch of the repository."
I think this does not align well with how we work today (without GitHub). E.g.
if a group decides to use GitHub to better track changes and maybe easily
integrate editorial comments, but decide to have all discussion on the mailing
list (and not use github issues), I think it would be fully okay for the
editors just commit directly based on the consensus on the mailing list. So I
think the important part is that substantial comments should only be committed
with wg consensus and therefore it is a good practice to have pull requests
first to declare consensus based on the concrete proposed edits. However, if a
different way to declare consensus is used that's fine as well. In short, I
think the SHOULD is to string here for the general case.

4) Sec 5.3 (mostly editorial):
"It is possible to use
   different processes for different documents in the Working Group.

   Working Group chairs SHOULD confirm that the Working Group has
   consensus to adopt any process.  In particular, the introduction of a
   more tightly-controlled process can have the effect of privileging
   positions already captured in documents, which might disadvantage
   alternative viewpoints."
This text seems to apply more general to all content in section 5 and should
maybe be moved out of section 5.3. Similarly I'm not sure if section 5.3.1 and
5.3.2 should actually be subsection of 5.3. The question of which issue to
track seem rather orthogonal to the question who resolves the issue (I agree
that this discussion does not make sense when no issue are tracked like in sec
5.1 but it could make sense for the mode in section 5.2)

5) Sec 5.3.1:
"The risk is that
   design changes might not always reflect the consensus of the Working
   Group."
I know that happens, even without GitHub, but should ideally not be the case.
However, I think the root cause is rather that changes were incorrectly not
identified by the editors as requiring consensus. Maybe this can be reworded
accordingly to not give the impression that it is okay to implement design
changes in a working group without consensus. Actually this is a broader
problem (independent of GitHub) because some editors, especially when they also
have been the authors of the individual draft, often implement design changes
first in a new version and then ask the working group for consensus. As I said
that's a different issue but it could be good to mention that GitHub and the
use of PRs can actually help to not use this practice but always each consensus
first.

6) Sec 5.3.1:
"... it is likely appropriate to move
   to a more tightly controlled process."
Maybe add something like "e.g. potentially during or after after working group
last call". I think in many groups (that do not use GitHub) it is very common
practice that editors have full control about the document. I mean I guess
that's their role. I think GitHub really can help to make the process more
transparent but it should not be a requirement. Especially shorter documents
that had have good consensus form the beginning, there might necessarily be a
need to every change that process. (I would say QUIC is rather the other
extreme.)

7) Sec 5.3.2:
"   Chairs might choose to manage the process of deciding which issues
   are substantive.  For instance, chairs might reserve the ability to
   use the "design" label to new issues (see Section 5.4.1) and to close
   issues marked as "design".  Chairs should always allow document
   editors to identify and address editorial issues as they see fit."
I guess you could also use normative language here: "MAY choose to" and "Chairs
SHOULD always allow"

8) Sec 5.3.2:
"   As documents mature further, explicit confirmation of technical
   decisions with the Working Group mailing list becomes more important."
Not sure I agree here. I know what you mean but explicit confirmation on the
mailing list is always important. Maybe there is a way to rephrase that (or
just remove that sentence...?)

9) Sec 5.3.2:
"   Gaining Working Group consensus about the resolution of issues can be
   done in the abstract, with editors being permitted to capture the
   outcome of discussions as they see fit."
I'm actually not sure what you mean here. Can you clarify?

10) Sec 5.3.2:
"   WGLC SHOULD be proposed as pull requests, and MUST be discussed on
   the mailing list, and MUST have chairs explicitly confirm consensus."
I agree with the "MUST be discussed on the mailing list" (as this is inline
with RFC2418 e.g. 3.2 but therefore doesn't necessarily need to be normative in
this doc) but find the other two SHOULD and MUST too strict. I don't think this
is aligned with the process we have today in groups and we should not use this
document to make the process more strict than it is. Especially I'm not sure
what "chairs MUST explicitly confirm consensus" really means and it seems to be
a requirement independent of GitHub and therefore eventually would even update
RFC2481.

11) Sec 5.4.4:
It might be too late for this kind of input, however, as I review the document
I'll note it here anyway. In taps we also have a "discuss" label to mark issue
that has been discussed but need further discussion e.g. at an (interim)
meeting. For this, one could even create a milestone to indicate at which
meeting more discussion should take place (or when e.g. a document is
text-ready until when text should be provided). We/the editors also did this
for a while in taps.

12) Section 6 seems to encourage revision only in preparation for meetings. I'm
not sure if that is inline with our usual working practice. I mean yes in
reality we see the draft submission deadline as forcing function for updates
and there want to keep it but we also do encourage to rev documents more often
and work continuously, e.g. when a mayor change was implemented or a mayor
issue resolved. And yes this works probably better for smaller documents but
the section seems a bit contradicting to this practice. I mean the reason that
we see many updates just at the draft deadline is because editors assign time
to make updates to resolve issue to match the deadline. If editors make
continuous changes we should encourage to update more often. Maybe it could
rather say something like: "Revisions SHOULD be submitted as I-Ds when a
signification issue has been resolved. Editors MAY bundle multiple changes in
one revision if updates are done in timely close coherence and SHOULD update at
least two weeks before any meeting." However, some of this advise is actually
not GitHub specific and might again just touch our general guidelines and as
such update RFC2026 and RFC2418...

13) sec 7:
"   Chairs MUST consider input from all discussion venues when assessing
   consensus including GitHub, mailing lists, interim meetings, and IETF
   meetings."
This seems like an update to RFC2481 again... as maybe also some other notes in
this section.

---

From: "Martin Thomson" <[email-masked]>
Date: Wed, 11 Mar 2020 17:50:17 +1100

Hi Mirja,

Thanks for taking the time to read through this.  I appreciate the input.

I've made some changes in https://github.com/ietf-gitwg/using-github/pull/45 but we might want to iterate on that some more.

On Tue, Mar 10, 2020, at 23:46, Mirja Kühlewind via Datatracker wrote:
> draft-ietf-git-github-wg-configuration. 

I think that document encompasses most of the stuff that I don't believe is appropriate for a BCP, so I think that the split is useful.  I think that others have addressed questions about the suitability of that document for publication.

> can provide a wrong signal if repos are created for some and not others.

The intention is to not be specific.  What I have heard, from multiple Working Group chairs, is that they have allowed people to create repos for individual contributions in the organization and that they - and their contributors - have found it to be valuable.  Chairs who do this seem to be quite open about allowing anyone to request a repo.  That seems like a healthy balance.

I think that is good, because it removes any implied status involved in having a draft repo in the org.  I personally have fewer issues with moving repos than Warren does, so maybe I don't mind keeping things private until they are adopted.  Other do report challenges though.

> vary from group to group, I think this really should be a SHOULD NOT only.

This has been iterated on since after Barry's review.  Do the changes in https://github.com/ietf-gitwg/using-github/pull/43 work here?

> think the SHOULD is to string here for the general case.

I think that it greatly improves transparency, and I want to add that WG policies could insist upon this practice, but we don't need normative force in the two statements in this paragraph.

> 4) Sec 5.3 (mostly editorial): [...]

The organization here is tricky, but I do agree that this text is generic.  I've moved the piece you quoted up to the intro to Section 5.

> use of PRs can actually help to not use this practice but always each consensus

I've added a piece about identifying the need for consensus, that is a good addition, but I don't see anything else actionable here.  I read the text as saying pretty much what you want it to.

> last call".

I don't want to prompt a particular outcome by making that sort of suggestion.  I appreciate that QUIC is weird, but it isn't THAT weird.  I've had several similar experiences in a relatively short period of time with HTTP/2, TLS, ACME, and a few others.

> I guess you could also use normative language here: "MAY choose to" and "Chairs

I don't like normative clauses in a for example, but the latter seems good.

> just remove that sentence...?)

This is really only emphasis, but I would prefer to keep it.

> I'm actually not sure what you mean here. Can you clarify?

You can get consensus on specific language (usually in the form of a PR in this setting) or you can gain consensus on the outline of a design, without text.  In the past, we've been quite effective at the latter as well as the former.  Specific text is more concrete, but it can be better (particularly for documents in early stages) to just work out a rough outline first.

> a requirement independent of GitHub and therefore eventually would even update

How about:

Ideally, substantive changes to documents that have passed WGLC are
proposed as pull requests, and MUST be discussed on the mailing list. Having
chairs explicitly confirm consensus on changes ensures that previous consensus
decisions are not overturned without cause.

> text-ready until when text should be provided). We/the editors also did this

We used labels like this in QUIC too.  Then we found that project boards were better for this. I think that someone suggested this during WGLC and we decided not to include it.

> such update RFC2026 and RFC2418...

The intent here is to reiterate 2418 regarding session documents and to encourage routine publication of snapshots.

> This seems like an update to RFC2481 again... as maybe also some other notes in

If you have concrete alternatives, I'm open to tweaking.  I think that as a restatement of existing principles, this is fine as is, but I'm sure that a different and better formulation is possible.

---

From: Mirja Kuehlewind <[email-masked]>
Date: Thu, 12 Mar 2020 19:27:14 +0100

Hi Martin,

See inline.

> On 11. Mar 2020, at 07:50, Martin Thomson <[email-masked]> wrote:
> 
> Hi Mirja,
> 
> Thanks for taking the time to read through this.  I appreciate the input..
> 

I think having a split would be fine (also there is a bit of a question if draft-ietf-git-github-wg-configuration needs to be published as RFC), however, this doesn’t seem like a clear split because there is a lot of redundant requirements in this docent in section 2 (as well as some sentences in 3) and I don’t see a value in stating it twice in two different documents (especially as this one seems to be the one that states it normatively). 

> I think that is good, because it removes any implied status involved in having a draft repo in the org.  I personally have fewer issues with moving repos than Warren does, so maybe I don't mind kee…

There is a risk if chairs allow repos for some draft but not other, that those drafts are seen as more important. So I think some more discussion is needed in this document that if a wg decide to support individual drafts in their GitHub organisation, they are expected to give that opportunity to everybody asking.

However, I still see some higher risk here for confusing for people who are less familiar with the IETF and the respective naming convention. In the datatracker the document for each wg are clearly separated into a section for wg doc and other related docs. That is less obvious on GitHub, especially for people who only find things over GitHub and are not aware of e.g. the datatracker or other IETF processes.

I’m not saying we should forbid this. Chairs are free in how they think they can manage their groups best. But I also think we should not recommend it. Also we should probably give more guidelines how to make it more clear that documents have different status in GitHub besides the draft name. E.g. there is the repo description that could include a “warning”. Or we could recommend to have all wg doc repos as pinned repos but other not…

> This has been iterated on since after Barry's review.  Do the changes in https://github.com/ietf-gitwg/using-github/pull/43 work here?

Yes that’s much better. Thanks!

> I think that it greatly improves transparency, and I want to add that WG policies could insist upon this practice, but we don't need normative force in the two statements in this paragraph.

Yes, it should not be normative but I’m also not sure if that practice is the right thing for every document. Yes it greatly can improve transparency and I agree that is is a good practice but for some e.g. small documents it might be more overhead/“process burden” than really needed. So I think it would be good to  reword slightly and/or add some more emphasis that there might be cases where it might not be needed.

>> maybe be moved out of section 5.3. Similarly I'm not sure if section 5..3.1 and

Thanks! What’s about 5.3.1 and 5.3.1. I would recommend to create a new subsection there and not have these subsubsection within 5.3.
> I've added a piece about identifying the need for consensus, that is a good addition, but I don't see anything else actionable here.  I read the text as saying pretty much what you want it to.

Thanks! I would also propose to slightly rephrase it to make it more GitHub specific 
"The risk is that changes that potentially contain design changes might get merged into the working version of the document before consensus of the Working Group is confirmed. However, transparency of these changes is expected to still be higher compared to when documents are maintained by the editors without any detailed change log.”

> I don't want to prompt a particular outcome by making that sort of suggestion.  I appreciate that QUIC is weird, but it isn't THAT weird.  I've had several similar experiences in a relatively short…

To be honest these groups are all “weird”, or rather to some extend similar. However, there are many groups that are much smaller scoped or at least work on less complicated protocols or only small protocol extensions. In all these cases the proposed tightly controlled process might be an over-kill (at least compared to how these group work currently without git). 

I don’t think this process is the best choice in general but rather one option. So I think it would be in general good to tone this part maybe a bit down. 

You can also replace “likely appropriate” with “may be useful” or something. My request is just to make this more a suggestion that a recommendation.

> I don't like normative clauses in a for example, but the latter seems good.

I suggested the MAY in the first sentence instead of might (which is not an example, no?)

> This is really only emphasis, but I would prefer to keep it.

I’m concern that this sentence implies that technical design changes do not need to be confirmed in the early phase, which I think it just wrong. I think this is more than emphasis.

> You can get consensus on specific language (usually in the form of a PR in this setting) or you can gain consensus on the outline of a design, without text.  In the past, we've been quite effective…

I believe you already revised this text based on other comments, right? (Lost track a bit). If so, maybe you can just point me add the next text…?

>>   the mailing list, and MUST have chairs explicitly confirm consensus.."

Yes, thanks!

> We used labels like this in QUIC too.  Then we found that project boards were better for this. I think that someone suggested this during WGLC and we decided not to include it.

Okay.

> The intent here is to reiterate 2418 regarding session documents and to encourage routine publication of snapshots.

Actually it might be better to not use normative language here and rather point to 2418. However, right now the text does not seems to encourage routine publication of snapshots but rather it seems to encourage to “only” publish if there is a meeting.

> If you have concrete alternatives, I'm open to tweaking.  I think that as a restatement of existing principles, this is fine as is, but I'm sure that a different and better formulation is possible.

I think it should not use normative text here and rather point to rfc2481 instead. Maybe: 

“As specified in RFC2481, chairs must consider input from all discussion venues when assessing consensus. These include mailing lists, IETF meetings, interim meetings as well as issues and discussion on GitHub.”

Thanks!
Mirja

---

From: "Martin Thomson" <[email-masked]>
Date: Fri, 13 Mar 2020 14:37:23 +1100

Hi Mirja,

This got long, so I trimmed a few things out where you had some useful suggestions that seemed obvious enough.  I have updated the same PR from before at https://github.com/ietf-gitwg/using-github/pull/45/files

On Fri, Mar 13, 2020, at 05:27, Mirja Kuehlewind wrote:
> There is a risk if chairs allow repos for some draft but not other, 
> that those drafts are seen as more important. So I think some more 
> discussion is needed in this document that if a wg decide to support 
> individual drafts in their GitHub organisation, they are expected to 
> give that opportunity to everybody asking.
> 
> However, I still see some higher risk here for confusing for people who 
> are less familiar with the IETF and the respective naming convention. 
> In the datatracker the document for each wg are clearly separated into 
> a section for wg doc and other related docs. That is less obvious on 
> GitHub, especially for people who only find things over GitHub and are 
> not aware of e.g. the datatracker or other IETF processes.
> 
> I’m not saying we should forbid this. Chairs are free in how they think 
> they can manage their groups best. But I also think we should not 
> recommend it. Also we should probably give more guidelines how to make 
> it more clear that documents have different status in GitHub besides 
> the draft name. E.g. there is the repo description that could include a 
> “warning”. Or we could recommend to have all wg doc repos as pinned 
> repos but other not…

I realize that this happens, but it is just one of many ways in which chairs might abuse their authority (intentionally or not). But I just don't see that happening here.  Chairs who have this policy are open about it and extend this opportunity to any who request it.

So I'm against making a firm recommendation.  I like the potential for there to be no special status implied by inclusion in an organization, any more than having a draft listed on https://tools.ietf.org/wg/$WG/ or https://datatracker.ietf.org/wg/$WG/documents/ implies any special status.

> Yes, it should not be normative but I’m also not sure if that practice 
> is the right thing for every document. Yes it greatly can improve 
> transparency and I agree that is is a good practice but for some e.g. 
> small documents it might be more overhead/“process burden” than really 
> needed. So I think it would be good to  reword slightly and/or add some 
> more emphasis that there might be cases where it might not be needed.

I'm fairly sure that we have consensus in the WG to keep a recommendation for this practice.  It is now a recommendation without normative force, are you suggesting that no such recommendation be made?  Because I don't think that is reflective of the practices that have emerged, nor is it a good idea.

> Thanks! What’s about 5.3.1 and 5.3.1. I would recommend to create a new 
> subsection there and not have these subsubsection within 5.3.

Maybe I'm not following you now.  My view is that the 4 paragraphs under 5.3 directly belong there without creating additional subsections.

> "The risk is that changes that potentially contain design changes might 
> get merged into the working version of the document before consensus of 
> the Working Group is confirmed. However, transparency of these changes 
> is expected to still be higher compared to when documents are 
> maintained by the editors without any detailed change log.”

I've reworded this, but I think that the second sentence you propose is duplicative of the paragraph that follows.  Here's what I have now:

> [...] The risk is from integrating changes including substantive decisions that don't reflect the
consensus of the Working Group or that the need for consensus on an issue is not identified.
>
> Changes made by editors under this process do not lack options for identifying and correcting problems.  GitHub and git provide tools for ensuring that changes are tracked and can be audited.

> I’m concern that this sentence implies that technical design changes do 
> not need to be confirmed in the early phase, which I think it just 
> wrong. I think this is more than emphasis.

I will have to disagree here.  The previous section is quite deliberate about the need for obtaining review and consensus on these decisions.  The point is that there are fewer opportunities left to find and discuss issues on a mature specification.

> I believe you already revised this text based on other comments, right? 
> (Lost track a bit). If so, maybe you can just point me add the next 
> text…?

You aren't the only one, it was in response to Alvaro:

> Chairs can declare Working Group consensus about the resolution of issues in the abstract, allowing editors discretion on how to capture the decisions in documents.
-- https://github.com/ietf-gitwg/using-github/pull/47/files#diff-f0832ea1138ac7354c220723b246954bR633-R636

> Actually it might be better to not use normative language here and 
> rather point to 2418. However, right now the text does not seems to 
> encourage routine publication of snapshots but rather it seems to 
> encourage to “only” publish if there is a meeting.

That happened as a result of a review (and now I'm losing track too): https://github.com/ietf-gitwg/using-github/pull/48/files

---

From: Rob Sayre <[email-masked]>
Date: Fri, 13 Mar 2020 13:44:33 +1000

On Fri, Mar 13, 2020 at 1:37 PM Martin Thomson <[email-masked]> wrote:

> That happened as a result of a review (and now I'm losing track too):
> https://github.com/ietf-gitwg/using-github/pull/48/files

This is all getting a bit long, to be honest. It is very odd for the IESG
to focus on this draft so intently. Dare I say "bikeshed"?

thanks,
Rob

---

From: "Martin Thomson" <[email-masked]>
Date: Fri, 13 Mar 2020 14:50:13 +1100

On Fri, Mar 13, 2020, at 14:44, Rob Sayre wrote:
> This is all getting a bit long, to be honest. It is very odd for the 
> IESG to focus on this draft so intently. Dare I say "bikeshed"?

I take the opposing view.  I greatly appreciate that the IESG care enough to make this document better.

But maybe this suggests that there is room for improvement in the process.  Maybe like using something like GitHub for all the minor stuff to reduce the number of stray email bits caught by innocent bystanders.  But the need for transparency at this stage of the process is greater than previously, so it's not that simple.

---

From: Rob Sayre <[email-masked]>
Date: Fri, 13 Mar 2020 13:58:32 +1000

> to make this document better.

Sure, but this level of scrutiny is not appropriate for a document that
contains easily-reversible decisions. It's not the same as shipping a
protocol that will become part of the internet.

But maybe this suggests that there is room for improvement in the process.
> Maybe like using something like GitHub for all the minor stuff to reduce
> the number of stray email bits caught by innocent bystanders.  But the need
> for transparency at this stage of the process is greater than previously,
> so it's not that simple.
>

I think the patience shown here is good, but don't think many of these
discussions are material. In particular, it's possible to run a pre-github
workflow in concert with the workflows in this draft.

thanks,
Rob