Subject: [Gendispatch] FW: New Version Notification for draft-kuehlewind-update-tag-04.txt

From: Mirja Kuehlewind <[email-masked]>
Date: Tue, 13 Jul 2021 13:02:08 +0000

Hi all,

as promised we revived draft-kuehlewind-update-tag and submitted a new version yesterday. There are no substantial changes but there are some editorial changes and a bit more text in the security consideration section:

- One part is about the need to have the same "kind" of consensus for amendments as the original RFC had. This question came recently up for me in another context and I added some thoughts.

- I tried to reflect some of the discussion about risks of failure. I personally believe that the worst thing that can happens is that this change will not improve the situation but I don't think it will make it worse. Therefore I would like to see this published as BCP. However, if people would be more comfortable to run this as an experiment that would be doable as well and we can discuss the parameters of that (duration and criteria for success/failure). More input on that point is definitely welcome!

Also we will update you in the next days with data on the analysis Suresh did about the usage of the Updates tag so far. So if that is of interest for you, please stay tuned and wait for another mail from us!

Mirja and Suresh

On 12.07.21, 16:23, "[email-masked]" <[email-masked]> wrote:

    A new version of I-D, draft-kuehlewind-update-tag-04.txt
    has been successfully submitted by Mirja Kuehlewind and posted to the
    IETF repository.

    Name:		draft-kuehlewind-update-tag
    Revision:	04
    Title:		Definition of new tags for relations between RFCs
    Document date:	2021-07-12
    Group:		Individual Submission
    Pages:		9
    URL:            https://www.ietf.org/archive/id/draft-kuehlewind-update-tag-04.txt
    Status:         https://datatracker.ietf.org/doc/draft-kuehlewind-update-tag/
    Html:           https://www.ietf.org/archive/id/draft-kuehlewind-update-tag-04.html
    Htmlized:       https://datatracker.ietf.org/doc/html/draft-kuehlewind-update-tag
    Diff:           https://www.ietf.org/rfcdiff?url2=draft-kuehlewind-update-tag-04

    Abstract:
       An RFC can include a tag called "Updates" which can be used to link a
       new RFC to an existing RFC.  On publication of such an RFC, the
       existing RFC will include an additional metadata tag called "Updated
       by" which provides a link to the new RFC.  However, this tag pair is
       not well-defined and therefore it is currently used for multiple
       different purposes, which leads to confusion about the actual meaning
       of this tag and inconsistency in its use.

       This document recommends the discontinuation of the use of the
       updates/updated by tag pair, and instead proposes three new tag pairs
       that have well-defined meanings and use cases.

    The IETF Secretariat

---

From: Suresh Krishnan <[email-masked]>
Date: Sat, 17 Jul 2021 01:24:52 -0400

Hi all,
  As Mirja mentioned in her mail, I have gone over 500 recent RFCs (8000-8500) to look at Updates relationships and characterize them. Out of these 500 there were 95 RFCs that updated other RFCs. Some of them updated multiple RFCs and the total number of updated RFCs was 163. Then I went about trying to classify the Updates relationship into the different kinds of update relationships described in our draft. Some of these were fairly easy to classify but others required a fair bit of reading to guess the intent of the authors/editors/WG (which by itself is a problem that needs to be solved). There were quite a few cases that were subjective and reasonable people could disagree on how to classify them. If you see an RFC that you think is miscategorized on the sheet (especially if you were an author/chair/AD) please do chime in.

Overall there was a fairly even split between the Amending relationship and the Extending relationship. 59 of the 95 RFCs ended up Amending the “Updated” RFC(s) and 50 of them ended up Extending the “Updated” RFCs (14 of the 95 RFCs ended up doing both). There were very few (4) updating RFCs that fell into the “See Also” category but that is probably explained by the fairly conservative use of Updates. For the “See Also” case it might be better to look at non-updating RFC (It would be a fairly time consuming exercise) to see if this tag is relevant.

Please let us know if you have any questions or comments. I have also uploaded the raw data to Google sheets in case you wanted to look (no account needed to view and comment) at

https://docs.google.com/spreadsheets/d/1OY33xNuMXrkxs8sQau6rHEzeIhKya0UfmiINMc1uCK8/edit?usp=sharing <https://docs.google.com/spreadsheets/d/1OY33xNuMXrkxs8sQau6rHEzeIhKya0UfmiINMc1uCK8/edit?usp=sharing>

Thanks
Suresh

> On Jul 13, 2021, at 9:02 AM, Mirja Kuehlewind <[email-masked]> wrote:
> 
> Hi all,
>

---

From: Brian E Carpenter <[email-masked]>
Date: Sun, 18 Jul 2021 09:19:30 +1200

Thanks Suresh, that was a big effort.

I have one question. Please just answer subjectively, I am not asking for 
more work, but do you have a feeling what % of the "Amended" are corrections vs changes of intention?

> There were very few (4) updating RFCs that fell into the “See Also” category

That doesn't surprise me. I think it's a useful relationship; for example 
a large number of the RFCs cited in "IPv6 node requirements" could have a 
"see also" relationship with each other, but it's quite different from "updates".

Regards
   Brian

On 17-Jul-21 17:24, Suresh Krishnan wrote:
> Hi all,
>   As Mirja mentioned in her mail, I have gone over 500 recent RFCs 
(8000-8500) to look at Updates relationships and characterize them. Out of these 500 there were 95 RFCs that updated other RFCs. Some of them updated multiple RFCs and the total number of updated RFCs was 163. Then I went about trying to classify the Updates relationship into the different kinds of update relationships described in our draft. Some of these were fairly easy to classify but others required a fair bit of reading to guess 
the intent of the authors/editors/WG (which by itself is a problem that needs to be solved). There were quite a few cases that were subjective and 
reasonable people could disagree on how to classify them. If you see an RFC that you think is miscategorized on the sheet (especially if you were an author/chair/AD) please do chime in.
> 
> Overall there was a fairly even split between the Amending relationship 
and the Extending relationship. 59 of the 95 RFCs ended up Amending the “Updated” RFC(s) and 50 of them ended up Extending the “Updated” RFCs (14 of the 95 RFCs ended up doing both). There were very few (4) updating RFCs that fell into the “See Also” category but that is probably explained by the fairly conservative use of Updates. For the “See Also” case it might be better to look at non-updating RFC (It would be a fairly time consuming exercise) to see if this tag is relevant.
>> On Jul 13, 2021, at 9:02 AM, Mirja Kuehlewind <[email-masked] <mailto:[email-masked]>> wrote:
>>
>> Hi all,
>>
>> as promised we revived draft-kuehlewind-update-tag and submitted a new 
version yesterday. There are no substantial changes but there are some editorial changes and a bit more text in the security consideration section:
>>
>> - One part is about the need to have the same "kind" of consensus for amendments as the original RFC had. This question came recently up for me 
in another context and I added some thoughts.
>> On 12.07.21, 16:23, "[email-masked] <mailto:[email-masked]>" <[email-masked] <mailto:[email-masked]>> wrote:
>>
>>
>>    Name:draft-kuehlewind-update-tag
>>    Revision:04
>>    Title:Definition of new tags for relations between RFCs
>>    Document date:2021-07-12
>>    Group:Individual Submission
>>    Pages:9
>>    URL:            https://www.ietf.org/archive/id/draft-kuehlewind-update-tag-04.txt <https://www.ietf.org/archive/id/draft-kuehlewind-update-tag-04.txt>
>>    Status:         https://datatracker.ietf.org/doc/draft-kuehlewind-update-tag/ <https://datatracker.ietf.org/doc/draft-kuehlewind-update-tag/>
>>    Html:           https://www.ietf.org/archive/id/draft-kuehlewind-update-tag-04.html <https://www.ietf.org/archive/id/draft-kuehlewind-update-tag-04.html>
>>    Htmlized:       https://datatracker.ietf.org/doc/html/draft-kuehlewind-update-tag <https://datatracker.ietf.org/doc/html/draft-kuehlewind-update-tag>
>>    Diff:           https://www.ietf.org/rfcdiff?url2=draft-kuehlewind-update-tag-04 <https://www.ietf.org/rfcdiff?url2=draft-kuehlewind-update-tag-04>
>>
>>    Abstract:

---

From: Suresh Krishnan <[email-masked]>
Date: Mon, 19 Jul 2021 00:10:09 -0400

Hi Brian,

> On Jul 17, 2021, at 5:19 PM, Brian E Carpenter <[email-masked]> wrote:
> 

I don’t recall exactly but I feel that there were more changes of intention than straight corrections as at least few of these were crypto changes that led to more stronger algorithm requirements.

> "see also" relationship with each other, but it's quite different from "updates".

Yep. Exactly. Fully agree,

Thanks
Suresh

---

From: "Martin Thomson" <[email-masked]>
Date: Mon, 19 Jul 2021 14:14:10 +1000

> Yep. Exactly. Fully agree,

Isn't that what citations are for?  Keeping metadata lean would be most consistent with its prominence.

---

From: Mark Nottingham <[email-masked]>
Date: Mon, 19 Jul 2021 14:23:20 +1000

I had the same reaction. Rather than just trying to figure out how people are using our existing, poorly-specified metadata and attempting to standardise that, we'd be better served by a holistic view of what the requirements for document metadata actually are, and how they can be served.

Cheers,

> On 19 Jul 2021, at 2:14 pm, Martin Thomson <[email-masked]> wrote:
> 
> -- 
> Gendispatch mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/gendispatch

--
Mark Nottingham   https://www.mnot.net/

---

From: Brian E Carpenter <[email-masked]>
Date: Mon, 19 Jul 2021 16:54:52 +1200

> Isn't that what citations are for?  Keeping metadata lean would be most consistent with its prominence.

Yes, in many cases a citation (and the implied "cited by") is enough, but "see also" is only intended for occasional use:
https://www.ietf.org/archive/id/draft-kuehlewind-update-tag-04.html#section-3-4

   Brian

---

From: Mirja Kuehlewind <[email-masked]>
Date: Mon, 19 Jul 2021 10:14:00 +0200

A citation gives you a reference from the new RFC to an existing one. If you also want to add a pointer from an existing RFC to a new one, you need to add meta data.

I don’t expect this will be used extensively as I do believe “amends" and “extents” covers the most important cases but it can still be useful (e.g. to connect a suite of recommendation documents…?).

> On 19. Jul 2021, at 06:54, Brian E Carpenter <[email-masked]> wrote:
>

---

From: Brian E Carpenter <[email-masked]>
Date: Tue, 20 Jul 2021 08:39:31 +1200

> A citation gives you a reference from the new RFC to an existing one. If you also want to add a pointer from an existing RFC to a new one, you need to add meta data.

Right, but I don't think the current text in the draft makes it clear when you would want to do this. Consider that "cited by" is already available in the tracker, e.g. https://datatracker.ietf.org/doc/rfc8200/referencedby/

I chose that example to make it clear that "See also" is not the same as "Cited by". Which of the RFCs that cite RFC8200 would deserve a "See also"? And, by the way, who decides, since this will be added after publication?

   Brian

---

From: Mirja Kuehlewind <[email-masked]>
Date: Tue, 20 Jul 2021 11:22:20 +0200

Please see inline.

> On 19. Jul 2021, at 22:39, Brian E Carpenter <[email-masked]> wrote:
> 
> I chose that example to make it clear that "See also" is not the same as "Cited by". Which of the RFCs that cite RFC8200 would deserve a "See also”?

Thanks for bring this example up. This list at least shows that “cited by” is not useful in this context. I guess most of the documents on this list “use” RFC8200. So for the reader of the new documents, it’s important to have the pointer to FC8200 but any reader of RFC8200 would probably not really be interested in most of those documents on this list. I don’t think I have a perfect answer which of these documents could be “see also” (and I didn’t try to put in the time just now to figure it out) but I would expect a much shorter list.

> And, by the way, who decides, since this will be added after publication?

I have an open issue for this on github:

https://github.com/mirjak/draft-kuehlewind-update-tag/issues/7

However, I on purpose did not try to address it in the last revision. I believe having these tags is just part of the consensus process as the rest of then document as well. As such similarly as for every other issue that is raised by the IESG, the IESG can block the document until the issue is addressed. Not sure we really need to say anything explicitly in the draft?

Mirja

---

From: Brian E Carpenter <[email-masked]>
Date: Fri, 23 Jul 2021 12:00:59 +1200

Below...
On 20-Jul-21 21:22, Mirja Kuehlewind wrote:
>>> A citation gives you a reference from the new RFC to an existing one. 
If you also want to add a pointer from an existing RFC to a new one, you need to add meta data.
> Thanks for bring this example up. This list at least shows that “cited by” is not useful in this context. I guess most of the documents on this list “use” RFC8200. So for the reader of the new documents, it’s important to have the pointer to FC8200 but 
any reader of RFC8200 would probably not really be interested in most of those documents on this list. I don’t think I have a perfect answer which of these documents could be “see also” (and I didn’t try to put in the time just now to figure it out) but I would expect a much shorter list.
> However, I on purpose did not try to address it in the last revision. I 
believe having these tags is just part of the consensus process as the rest of then document as well. As such similarly as for every other issue that is raised by the IESG, the IESG can block the document until the issue is addressed. Not sure we really need to say anything explicitly in the 
draft?

"See also" is a bit different because it doesn't relate to normative effects in the way that "Amends" and "Extends" do. So it could affect any stream, not just IESG-approved documents. I guess that means that the RFC Editor is responsible. The old "Also RFCxxxx" designation that you find in the RFC index, starting with RFC29 and continuing at least through RFC2389, was certainly an RFC Editor decision. Possibly all of those should be changed to "See also"?

   Brian

---

From: Mirja Kuehlewind <[email-masked]>
Date: Fri, 23 Jul 2021 11:02:47 +0200

Hi Brian,

On this point:

> On 23. Jul 2021, at 02:00, Brian E Carpenter <[email-masked]> wrote:
> 
>>> 
> "See also" is a bit different because it doesn't relate to normative effects in the way that "Amends" and "Extends" do. So it could affect any stream, not just IESG-approved documents. I guess that means that the RFC Editor is responsible. The old "Also RFCxxxx" designation that you find in the RFC index, starting with RFC29 and continuing at least through RFC2389, was certainly an RFC Editor decision. Possibly all of those should be changed to "See also”?

I didn’t consider the RFC editor as an option here yet, also because so far this proposal is scoped to the IETF stream. However, I also don’t see a need to strive for completeness for the “See also” tags. My assumption is rather to be conservative in adding it because if you add it too broadly it because less useful. Therefore having the authors or wg t decide about the “see also” tag is fine for me.

The draft explicitly discusses that there is no intention to change tags for existing drafts, as, to say it simply, that might be more effort that it’s worth it.

Mirja

---

From: Brian E Carpenter <[email-masked]>
Date: Sat, 24 Jul 2021 08:47:21 +1200

On 23-Jul-21 21:02, Mirja Kuehlewind wrote:
>> believe having these tags is just part of the consensus process as the 
rest of then document as well. As such similarly as for every other issue 
that is raised by the IESG, the IESG can block the document until the issue is addressed. Not sure we really need to say anything explicitly in the 
>> draft?
>>
>> "See also" is a bit different because it doesn't relate to normative effects in the way that "Amends" and "Extends" do. So it could affect any stream, not just IESG-approved documents. I guess that means that the RFC 
Editor is responsible. The old "Also RFCxxxx" designation that you find in the RFC index, starting with RFC29 and continuing at least through RFC2389, was certainly an RFC Editor decision. Possibly all of those should be changed to "See also”?
> The draft explicitly discusses that there is no intention to change tags for existing drafts, as, to say it simply, that might be more effort that it’s worth it.

In general, I agree. The "Also RFCxxxx" anomaly in the RFC index has always annoyed me, because it is quite different from "Also BCPxxxx", "Also STDxxxx" and the old "Also FYIxxxx". If we do officially create "See also" 
I will strongly recommend the RFC Editor to change all the old "Also RFCxxxx" to "See also". But that is out of scope for your draft; sorry for the distraction.

   Brian

---

From: Eric Rescorla <[email-masked]>
Date: Wed, 28 Jul 2021 09:59:06 -0700

Unfortunately, I cannot be at the meeting....

On Fri, Jul 16, 2021 at 10:25 PM Suresh Krishnan <[email-masked]>
wrote:

> (8000-8500) to look at Updates relationships and characterize them.
>

Thanks for doing this.

Out of these 500 there were 95 RFCs that updated other RFCs. Some of them
> updated multiple RFCs and the total number of updated RFCs was 163. Then I
> went about trying to classify the Updates relationship into the different
> kinds of update relationships described in our draft. Some of these were
> fairly easy to classify but others required a fair bit of reading to guess
> the intent of the authors/editors/WG (which by itself is a problem that
> needs to be solved). There were quite a few cases that were subjective and
> reasonable people could disagree on how to classify them
>

I think this is the key point, along with the fairly high degree of overlap
(about 15 which both extend and amend). In my experience on the IESG, a lot
of time was spent trying to determine precisely what tags should appear,
far out of proportion to the actual value provided by those tags. Rather
than try to draw ever finer distinctions, which seems like it will make
that problem worse, I would instead propose we simply deprecate this all in
favor of "See Also" (or no tags at all).

-Ekr

. If you see an RFC that you think is miscategorized on the sheet
> (especially if you were an author/chair/AD) please do chime in.
>
> and the Extending relationship. 59 of the 95 RFCs ended up Amending the
> “Updated” RFC(s) and 50 of them ended up Extending the “Updated” RFCs (14
> of the 95 RFCs ended up doing both). There were very few (4) updating RFCs
> that fell into the “See Also” category but that is probably explained by
> the fairly conservative use of Updates. For the “See Also” case it might be
> better to look at non-updating RFC (It would be a fairly time consuming
> exercise) to see if this tag is relevant.
>
> Please let us know if you have any questions or comments. I have also
> uploaded the raw data to Google sheets in case you wanted to look (no
> account needed to view and comment) at
>
>
> https://docs.google.com/spreadsheets/d/1OY33xNuMXrkxs8sQau6rHEzeIhKya0UfmiINMc1uCK8/edit?usp=sharing
>
> Thanks
> Suresh
>
> version yesterday. There are no substantial changes but there are some
> editorial changes and a bit more text in the security consideration section:
>
> - One part is about the need to have the same "kind" of consensus for
> amendments as the original RFC had. This question came recently up for me
> - I tried to reflect some of the discussion about risks of failure. I
> personally believe that the worst thing that can happens is that this
> change will not improve the situation but I don't think it will make it
> worse. Therefore I would like to see this published as BCP. However, if
> people would be more comfortable to run this as an experiment that would be
> doable as well and we can discuss the parameters of that (duration and
> criteria for success/failure). More input on that point is definitely
> welcome!
>
> Also we will update you in the next days with data on the analysis Suresh
> did about the usage of the Updates tag so far. So if that is of interest
> for you, please stay tuned and wait for another mail from us!
>
> Mirja and Suresh
>
>
>
>
>    URL:
> https://www.ietf.org/archive/id/draft-kuehlewind-update-tag-04.txt
>    Status:
> https://datatracker.ietf.org/doc/draft-kuehlewind-update-tag/
>    Html:
> https://www.ietf.org/archive/id/draft-kuehlewind-update-tag-04.html
>    Htmlized:
> https://datatracker.ietf.org/doc/html/draft-kuehlewind-update-tag
>    Diff:
> https://www.ietf.org/rfcdiff?url2=draft-kuehlewind-update-tag-04
>
>    Abstract:

---

From: Mirja Kuehlewind <[email-masked]>
Date: Wed, 28 Jul 2021 17:21:46 +0000

Just to clarify one small point: those 15 that amend and extend, usually do that for different RFCs.

From: Eric Rescorla <[email-masked]>
Date: Wednesday, 28. July 2021 at 19:01
To: Suresh Krishnan <[email-masked]>
Cc: Mirja Kuehlewind <[email-masked]>, rfc-interest <[email-masked]>, GENDISPATCH List <[email-masked]>
Subject: Re: [Gendispatch] Some data about Updates in recent RFCs (was Re: New Version Notification for draft-kuehlewind-update-tag-04.txt)

On Fri, Jul 16, 2021 at 10:25 PM Suresh Krishnan <[email-masked]<mailto:[email-masked]>> wrote:
Hi all,
  As Mirja mentioned in her mail, I have gone over 500 recent RFCs (8000-8500) to look at Updates relationships and characterize them.

Thanks for doing this.

Out of these 500 there were 95 RFCs that updated other RFCs. Some of them updated multiple RFCs and the total number of updated RFCs was 163. Then I went about trying to classify the Updates relationship into the different kinds of update relationships described in our draft. Some of these were fairly easy to classify but others required a fair bit of reading to guess the intent of the authors/editors/WG (which by itself is a problem that needs to be solved). There were quite a few cases that were subjective and reasonable people could disagree on how to classify them

I think this is the key point, along with the fairly high degree of overlap (about 15 which both extend and amend). In my experience on the IESG, a lot of time was spent trying to determine precisely what tags should appear, far out of proportion to the actual value provided by those tags. Rather than try to draw ever finer distinctions, which seems like it will make that problem worse, I would instead propose we simply deprecate this all in favor of "See Also" (or no tags at all).

-Ekr

. If you see an RFC that you think is miscategorized on the sheet (especially if you were an author/chair/AD) please do chime in.

https://docs.google.com/spreadsheets/d/1OY33xNuMXrkxs8sQau6rHEzeIhKya0UfmiINMc1uCK8/edit?usp=sharing<https://protect2.fireeye.com/v1/url?k=0f3fb66f-50a48ebd-0f3ff6f4-866132fe445e-1daaf7c0173ab43d&q=1&e=c3259175-f2aa-4da4-9d8d-7ef2fd8453a9&u=https%3A%2F%2Fdocs.google.com%2Fspreadsheets%2Fd%2F1OY33xNuMXrkxs8sQau6rHEzeIhKya0UfmiINMc1uCK8%2Fedit%3Fusp%3Dsharing>

Thanks
Suresh

On Jul 13, 2021, at 9:02 AM, Mirja Kuehlewind <[email-masked]<mailto:[email-masked]>> wrote:

Hi all,

Mirja and Suresh

On 12.07.21, 16:23, "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>> wrote:

   Abstract:

   The IETF Secretariat

--
Gendispatch mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/gendispatch

---

From: Stephen Farrell <[email-masked]>
Date: Wed, 28 Jul 2021 19:58:58 +0100

Hiya,

On 28/07/2021 17:59, Eric Rescorla wrote:
> In my experience on the IESG, a lot

I agree with ekr on the above. There is another option,
with which I'm fine, but that seems to bug a lot of
IETFers - we could ignore the problem and leave things
in their current imperfect state.

Cheers,
S.

---

From: Eric Rescorla <[email-masked]>
Date: Wed, 28 Jul 2021 12:02:48 -0700

> in their current imperfect state.

I could also live with this, especially if we tacitly agreed to chill out
about the contents of "Updates" :)

-Ekr