Subject: Re: [arch-d] [IAB] Review of: draft-iab-protocol-transitions-05

From: Dave Thaler <[email-masked]>
Date: Tue, 21 Feb 2017 23:38:55 +0000

Thanks Dave for the detailed review!

I have accepted the editorial nits, and made other changes as noted below.

    The document is well-organized and well-written.  There are some clarifications and expansions worth considering, as noted below, and some basic points cited here:

    The document tends to merely mention essential issues, such as incentives, without giving much insight into either methods for adequately assessing incentives or deciding how to consider them in protocol design.

Unfortunately we may not have a lot of experience with such methods.  However, I have made several notable changes in response to this and others’ comments.

An Extensibility section now discusses extensibility in more detail and refers to the IAB’s recent RFC 6709 regarding protocol design.

A Conclusions section now encourages the community to share such insights and future experiences, on the architecture-discuss list.

    The document also seems to conflate "adoption" with "transition".

Much of the content of the document applies to initial adoption of a protocol, as well as to later transitions to revisions.  While transitions carry significant additional burdens, beyond initial adoption, the IETF needs attention to initial adoption issues every bit as much as it needs attention to transition issues.

Added terminology point to the Introduction saying that transition is generic, with four types of transition (adoption of a new protocol with no precedent, transitioning from an old to a new protocol, transitioning from an old to a new version of a protocol, and decommissioning of an obsolete protocol).  Of course some points in the doc apply more strongly to certain types of transition.

    Although the document references open source implementations and the challenges of having a timeline, it should emphasize the role of the former more and the severe problems with the latter.

I rewrote the section about timelines and criteria for phrases based on your feedback.

    Might be worth adding some examples of highly successful transitions.  MIME is, predictably, my favorite example.  There had been multiple attempts to replace existing, text-based email with a new version that supported multi-media.  MIME instead created an overlay that required no change to the infrastructure.

When the discussed it, the consensus was that additional case studies in the doc were not necessary.  Unless of course there’s a key point that would be called out that isn’t captured at all in the other case studies.

    These above suggestions are in line with Eliot's call for more 'meat'.  The document touches on essential issues.  But for the IETF to deal with the issues well, there needs to be more detailed basis giving guidance for how to attend to them. This is particularly important for issues such as incentives analysis and aligning to incentives, since they are topics not normally within the purview of Internet engineers.

(If the feeling is that the meat should be added via later documents then there should at least, now, be development of some plan for those

documents.)

    Stewart's call for considering the requirement of transition considerations -- I'd suggest 'adoption considerations' -- would press working groups to do far more due diligence about the barriers to adoption that is typically done now.

Perhaps, but that’s feedback for the IESG to consider, rather than the IAB’s purview per se.

Detailed:

I believe I’ve incorporated your various detailed points into the new draft, soon to be posted…

BTW, the case studies were written by others (I’m the editor) and so there may still be
additional changes after that to address some of your points in the IPv6 case study.

Dave

---

From: Dave Crocker <[email-masked]>
Date: Tue, 21 Feb 2017 15:54:38 -0800

On 2/21/2017 3:38 PM, Dave Thaler wrote:
>     The document tends to merely mention essential issues, such as
> incentives, without giving much insight into either methods for
> adequately assessing incentives or deciding how to consider them in
> protocol design.
>
>
>
> Unfortunately we may not have a lot of experience with such methods.
> However, I have made several notable changes in response to this and
> others’ comments.

Dave,

So, my primary concern is that when something is mentioned, it is made 
sufficiently real to the reader, so there is some shared substance to 
their understanding, among the range of readers we expect for the document.

If the best shared understanding that is possible is that we don't 
understand this item well, well, that at least is honest (and, in an odd 
way, thorough.)

>     Might be worth adding some examples of highly successful
> transitions.  MIME is, predictably, my favorite example.  There had been
> multiple attempts to replace existing, text-based email with a new
> version that supported multi-media.  MIME instead created an overlay
> that required no change to the infrastructure.
>
>
>
> When the discussed it, the consensus was that additional case studies in
> the doc were not necessary.  Unless of course there’s a key point that
> would be called out that isn’t captured at all in the other case studies.

Here's the catch:  (similar to my above point) Who is the document for?

A paper like this, without diligent examples, is purely conceptual.  A 
typical reader will not have enough background to be able to move from 
the abstract text of the document to the real-world design and planning 
efforts that they do.  Examples are not just to explain the concepts, 
but to tie them to pragmatics for which a reader is likely to have a 
background.

>     These above suggestions are in line with Eliot's call for more
> 'meat'.  The document touches on essential issues.  But for the IETF to
> deal with the issues well, there needs to be more detailed basis giving
> guidance for how to attend to them. This is particularly important for
> issues such as incentives analysis and aligning to incentives, since
> they are topics not normally within the purview of Internet engineers.
>
> (If the feeling is that the meat should be added via later documents
> then there should at least, now, be development of some plan for those
>
> documents.)
>
>
>
>     Stewart's call for considering the requirement of transition
> considerations -- I'd suggest 'adoption considerations' -- would press
> working groups to do far more due diligence about the barriers to
> adoption that is typically done now.
>
>
>

In terms of making requirements for the IETF stream, sure.  But in terms 
of pressing the benefit of the exercise, no matter who is doing the work 
or under whose aegis, no.  The latter is entirely reasonable for the IAB 
to describe and promote...

> draft, soon to be posted…

Thanks for the thoughtful response and (of course) for the doc 
modifications...

d/

-- 
Dave Crocker
Brandenburg InternetWorking
bbiw.net