Subject: [Rfced-future] Who are the customers?

From: Russ Housley <[email-masked]>
Date: Mon, 4 Jan 2021 18:04:00 -0500

Looking at RFC 8728, there is an expectation that the RSE should see the stream manager as customers. RFC 8728, s2.1: "The RSE is expected to cooperate closely with the IETF LLC and the stream managers."  I always assumed that the IETF LLC was included here because that is the legal entity that holds the contract, not because they represent an additional customer.

In addition, RFC 8728, s2.1.2.1.2 says that the RSE makes policies that are good for the "broader Internet community".  However, I do not think we have reached consensus on the topic of "customers".  We do not have a way to represent the broader community as an RFC Series customer.  The idea of a NomCom selected at-large person has been suggested, but I do not think that is not going to make it better.

Russ

---

From: Brian E Carpenter <[email-masked]>
Date: Tue, 5 Jan 2021 13:32:46 +1300

> In addition, RFC 8728, s2.1.2.1.2 says that the RSE makes policies that are good for the "broader Internet community".  However, I do not think we have reached consensus on the topic of "customers"…

Agreed, that is more of a mechanism to reduce the chances of groupthink.

I think the best thing we can do at the level of process & structure is what we already seem to agree about - having an open, anyone-can-join discussion forum. We could of course do things like open, anyone-can-answer surveys when there are specific questions to be answered but that's an operational matter. How the discussions & surveys are made known to the wider community also seems like an operational matter, as part of outreach in general.

Probably we should define clearly who's responsible for such outreach.

    Brian

---

From: John C Klensin <[email-masked]>
Date: Mon, 04 Jan 2021 21:02:29 -0500

--On Tuesday, January 5, 2021 13:32 +1300 Brian E Carpenter
<[email-masked]> wrote:

>> Looking at RFC 8728, there is an expectation that the RSE
>> should see the stream manager as customers. RFC 8728, s2.1:
>> "The RSE is expected to cooperate closely with the IETF LLC
>> and the stream managers."  I always assumed that the IETF LLC
>> was included here because that is the legal entity that holds
>> the contract, not because they represent an additional
>> customer.
>> 
>> In addition, RFC 8728, s2.1.2.1.2 says that the RSE makes
>> policies that are good for the "broader Internet community".
>> However, I do not think we have reached consensus on the
>> topic of "customers".  We do not have a way to represent the
>> broader community as an RFC Series customer.  The idea of a
>> NomCom selected at-large person has been suggested, but I do
>> not think that is not going to make it better.
> 
> I think the best thing we can do at the level of process &
> structure is what we already seem to agree about - having an
> open, anyone-can-join discussion forum. We could of course do
> things like open, anyone-can-answer surveys when there are
> specific questions to be answered but that's an operational
> matter. How the discussions & surveys are made known to the
> wider community also seems like an operational matter, as part
> of outreach in general.
> 

Brian,

While I agree with all of that, I think it is important to come
back to Russ's question, which is one that has been a problem
for this effort for well over a year now.   While I don't think
it is helpful to try to prioritize "customers" or what we might
call stakeholders in another context, there are at least three
groups of users of the series we tend to forget about (or
excessively de-emphasize) as we try to sort through issues of
representation, questions to be answered when we think about
surveys, and particular needs.

Two examples:

(1) Despite our best efforts, and often a good degree of
consensus within the IETF, sometimes we produce specifications
that either do not work or that fail to get traction.  In some
cases, the reason, at least in retrospect, is that we solved a
problem that no one cared about or that was just overtaken by
events.  In others, we just get it wrong or discover that we
solved the wrong problem.  Especially in the latter cases, and
also when the consensus turns out to be somewhat rough, it is
important to have good records of different ways that the issues
were thought about, different proposed solutions, etc., so that,
if we need to come back to the topic(s), we have the potential
for getting a head start and not reinventing the previous style
of wheel.  While, as things are defined today, this mostly
interacts with the Independent Stream, I note that (regardless
of what the authors intended) RFC 8789 does not prohibit
publishing a document with which the IETF participant  community
is in violent disagreement as long as there is IETF consensus
that it should be published.

(2) Part of the use of the RFC Series is to publish standards,
as distinct from, e.g., evolving engineering specifications.  A
key audience for standards are people using them in procurement
specifications and expecting that assertions of conformance on
the part of those producing products will be meaningful, stable
enough to outlast procurement cycles, and helpful in predicting
interoperability and interchangeability. Precisely because we
sometimes lose sight of that distinction and its importance, it
is important that the RFC Editor function (as well as relevant
streams and the IANA function) be sensitive to the issues
involved and the constraints they put on the style and
presentation of publications.

We have had, and will probably continue to have, difficulty
getting good direct representation of those interests.  Even if
we could find a token procurement person or two (of the
bean-counting subspecies or otherwise) and get them into a WG,
panel, board, or survey population, it is not clear that we are
good at listening or putting ourselves into their positions and
perspectives.  However, that does not make the perspective and
requirements any less important.

best,
   john

---

From: Christian Huitema <[email-masked]>
Date: Mon, 4 Jan 2021 22:06:38 -0800

> presentation of publications.

John,

I don't think that the RFC edition function can have any meaningful 
influence on the creation of documents "stable enough to outlast 
procurement cycles". Documents do not become unstable, the technology 
that they described becomes obsolete. It is often supplanted by the 
arrival of a new tech, often documented in new series of RFC. Do you see 
a future in which the RFC Edition function might refuse to publish new 
documents by fear of making previously published RFC "unstable"? I 
don't. Besides, competition from new tech is not the only reason for 
instability. We also have to account for deployment experience, 
unforeseen security issues, or changes in the technology landscape. This 
is not something that the RFC Edition can influence.

-- Christian Huitema

---

From: John C Klensin <[email-masked]>
Date: Tue, 05 Jan 2021 14:17:55 -0500

--On Monday, January 4, 2021 22:06 -0800 Christian Huitema
<[email-masked]> wrote:

>> as distinct from, e.g., evolving engineering specifications.
>> A key audience for standards are people using them in
>> procurement specifications and expecting that assertions of
>> conformance on the part of those producing products will be
>> meaningful, stable enough to outlast procurement cycles, and
>> helpful in predicting interoperability and
>> interchangeability. Precisely because we sometimes lose sight
>> of that distinction and its importance, it is important that
>> the RFC Editor function (as well as relevant streams and the
>> IANA function) be sensitive to the issues involved and the
>> constraints they put on the style and presentation of
>> publications.
> 
> John,
> 
> I don't think that the RFC edition function can have any
> meaningful influence on the creation of documents "stable
> enough to outlast procurement cycles". Documents do not become
> unstable, the technology that they described becomes obsolete.
> It is often supplanted by the arrival of a new tech, often
> documented in new series of RFC. Do you see a future in which
> the RFC Edition function might refuse to publish new documents
> by fear of making previously published RFC "unstable"? I
> don't. Besides, competition from new tech is not the only
> reason for instability. We also have to account for deployment
> experience, unforeseen security issues, or changes in the
> technology landscape. This is not something that the RFC
> Edition can influence.

Christian,

I agree with what you write above, including your conclusion.  

However, there have been fairly recent proposals that we make
incremental changes to RFCs rather than creating new documents
and that we integrate approved errata into the text of RFCs.
Even now, we make "HTML with inline errata" available as one of
the formats in the RFC Editor's repository.  I would expect an
RSx to have sufficient knowledge to take a professionally
competent position, and be willing to educate others, about the
issues such proposals might raise relative to what I wrote
above.  I also think it is important that any oversight
committee or process have participants who would be able and
willing to understand those issues and the tradeoffs involved
and, ideally, that it would have members who would notice and
identify those issues if the RSx failed to do so.

Does that make the interaction with the RFC Editor Function and
what it can influence more clear?

best,
  john

---

From: Brian E Carpenter <[email-masked]>
Date: Wed, 6 Jan 2021 09:11:35 +1300

Excuse front-posting but I think I'm setting off in yet another
slightly different direction.

Do we even *know* in any meaningful way who reads RFCs apart from
people active in the existing four streams?

Yes, we assume that the readers include operators, implementers,
procurement departments, government officials, academics etc., but we
don't have a real analysis; we don't know how many people this means
and we certainly don't have a mailing list. (FYI there are 367 email
addresses subscribed to rfc-interest, which means that it is clearly a
small fraction of the customers. Although I would love to know who
[email-masked], [email-masked] and their numerical friends are.)

I feel that before we make too many assumptions about how to represent
the customers, we need some actual facts about who they are and how
we could talk to them. Getting those facts might cost some momey and
effort, but I think it would be justified.

Regards
   Brian

---

From: Brian Rosen <[email-masked]>
Date: Tue, 5 Jan 2021 15:57:31 -0500

<as individual>
I don’t think it’s that hard.

Example:
I work on the emergency call system, 9-1-1 in the U.S.  There is a new system, Next Generation 9-1-1, which is based on a number of IETF RFCs including the SIP RFCs (3261,…) and several specific emergency call RFCs such as RFC6881, RFC4119, RFC5222.

There is a technical standard, NENA-STA-010, which has a normative reference to these IETF standards.  I work with states deploying this system.  They have RFPs that require adherence to NENA-STA-010, and often expressly require support of the referenced standards.  Vendors respond to the RFPs and commit to various levels of conformance.  They get into contracts that commit them to conformance.

So the customers are the state government entities, although they don’t read the RFCs initially, and the vendors, who do.  I have been specifically involved with verification and validation studies that are commissioned by a state to determine if the winning vendor’s software meets the contract requirements.  Sometimes we do find cases where they don’t implement the standard, and we point that out to the state.  The state then decides if they believe that constitutes a violation of the contract.  Usually, they negotiate some kind of “cure”, mostly time to come into conformance.   

That’s a pretty clear case of how an RFC, and conformance to it, gets into contracts, and testing, and potentially legal wrangling (although the latter hasn’t occurred on an RFC compliance issue in my experience yet).  None of the entire universe I’ve described, other than me, typically participates in the IETF process.  Some (vendors) read the documents, and implement them.

Brian

> On Jan 5, 2021, at 3:11 PM, Brian E Carpenter <[email-masked]> wrote:
> 
> -- 
> Rfced-future mailing list
> [email-masked]
> https://www.iab.org/mailman/listinfo/rfced-future

---

From: John C Klensin <[email-masked]>
Date: Tue, 05 Jan 2021 16:17:04 -0500

--On Wednesday, January 6, 2021 09:11 +1300 Brian E Carpenter
<[email-masked]> wrote:

> Do we even *know* in any meaningful way who reads RFCs apart
> from people active in the existing four streams?
> 
> Yes, we assume that the readers include operators,
> implementers, procurement departments, government officials,
> academics etc., but we don't have a real analysis; we don't
> know how many people this means and we certainly don't have a
> mailing list. (FYI there are 367 email addresses subscribed to
> rfc-interest, which means that it is clearly a small fraction
> of the customers. Although I would love to know who
> I feel that before we make too many assumptions about how to
> represent the customers, we need some actual facts about who
> they are and how we could talk to them. Getting those facts
> might cost some momey and effort, but I think it would be
> justified.

Brian,

While I am very sympathetic to you desire for that information,
there are at least two problems with trying to get those data of
which you (and we) should be aware).

(1) Even for rfc-interest, it is easy to tell how many people
are subscribed, but hard or impossible to tell how many are
actually reading it, much less to characterize the profiles of
the latter.  In particular, remembering that we require that
people sign up for a list to post to it, which can (and
occasionally demonstrably has) resulted in people signing up to
some IETF lists to post a one-time comment (often good) or to
spam the list (almost never good), there may be such signups
there.  Possibly relevant, I assume I'm not the only person who
is at least fairly active in the IETF and on this list, who is
subscribed for digests, and who might actually read them a
couple of times a month.

(2) The procurement people I'm concerned about typically use and
cite RFCs but it is very rare that they actually read them.
Instead, they are typically told by others (sometimes even us)
which RFCs describe a particular set of requirements and they
dutifully write them into contracts.  Only if something goes
wrong that appears to be a problem --one for which the
specifications in those RFCs would seem to be applicable-- do
people in their sphere of influence and authority typically
start reading.  Those "people" are often lawyers and those
supporting them.   That does not make the procurement types any
less important as users of/ customers for RFCs.  It makes having
the specs clear and precise enough that it is possible to tell
what is supposed to be conforming behavior and what is not (as
Christian points out, that is not a problem the RFC Editor can
affect very much... and we have steadily reduced how much over
the last decade or two) but also that they documents themselves
and their presentation be stable enough that there is minimal
possibility of disputes over, or having to do estensive
detective work to infer, exactly when the specification was at
the time contracts were signed or went into effect.

    john

---

From: Stephen Farrell <[email-masked]>
Date: Tue, 5 Jan 2021 21:22:58 +0000

Hiya,

On 05/01/2021 20:11, Brian E Carpenter wrote:
> Getting those facts might cost some momey and

If we do need to know more about that (*), then
wouldn't that really be a task for an incoming
RSE and not this group?

Cheers,
S.

(*) I'm not really convinced yet - I do think we
know enough about who those readers are, we just
don't know if they care about all the stuff we're
talking about here;-) And I'm not sure that's
a thing we could usefully find out about TBH.

---

From: John C Klensin <[email-masked]>
Date: Tue, 05 Jan 2021 16:24:17 -0500

Brian,

An excellent and specific example of what I was trying to
describe in more general terms in a note that crossed yours in
the email.  The situation you describe is a little special (but
typical of procurements and contracts in regulatory/ regulated
environments) because a third party, outside the organization
structure of the procurement person or department, is
determining what RFCs are required and referenced.  In a more
typical commercial environment, it is the RFCs that are
referenced rather than an intermediate industry or regulatory
standard (not developed by the IETF) that, in turn, references
them. 

I don't think that, in the end, that distinction is very
important to the understanding Brian Carpenter is looking for.
Hence +1 to what you wrote.

   john

--On Tuesday, January 5, 2021 15:57 -0500 Brian Rosen
<[email-masked]> wrote:

> <as individual>
> I don't think it's that hard.
> 
> Example:
> I work on the emergency call system, 9-1-1 in the U.S.  There
> is a new system, Next Generation 9-1-1, which is based on a
> number of IETF RFCs including the SIP RFCs (3261,…) and
> several specific emergency call RFCs such as RFC6881, RFC4119,
> RFC5222.
> 
> There is a technical standard, NENA-STA-010, which has a
> normative reference to these IETF standards.  I work with
> states deploying this system.  They have RFPs that require
> adherence to NENA-STA-010, and often expressly require support
> of the referenced standards.  Vendors respond to the RFPs and
> commit to various levels of conformance.  They get into
> contracts that commit them to conformance.
> 
> So the customers are the state government entities, although
> they don't read the RFCs initially, and the vendors, who do.
> I have been specifically involved with verification and
> validation studies that are commissioned by a state to
> determine if the winning vendor's software meets the
> contract requirements.  Sometimes we do find cases where they
> don't implement the standard, and we point that out to the
> state.  The state then decides if they believe that
> constitutes a violation of the contract.  Usually, they
> negotiate some kind of "cure", mostly time to come into
> conformance.   
> 
> That's a pretty clear case of how an RFC, and conformance to
> it, gets into contracts, and testing, and potentially legal
> wrangling (although the latter hasn't occurred on an RFC
> compliance issue in my experience yet).  None of the entire
> universe I've described, other than me, typically
> participates in the IETF process.  Some (vendors) read the
> documents, and implement them.
> 
> Brian

---

From: Brian E Carpenter <[email-masked]>
Date: Wed, 6 Jan 2021 10:31:28 +1300

> wouldn't that really be a task for an incoming

Except that at the moment we have no idea how best to
represent the requirements and issues of those customers
in the "open" strategy discussion. In that sense, there
are empty seats at the table. (And I am not disagreeing
with John Klensin's or Brian Rosen's comments at all.
Just observing that the majority of the customers simply
aren't represented in our discussions.)

   Brian

---

From: Stephen Farrell <[email-masked]>
Date: Tue, 5 Jan 2021 21:42:24 +0000

Hiya,

> aren't represented in our discussions.)

Sure. (As an aside, I'd prefer call them readers and not
customers - the latter seems to me to have an unintended
assumption that all readers that count are doing something
commercial, and I don't think that's true.)

I still think that finding out about all that, and maybe
proposing what, if anything to do about the results, seems
like a fine thing for an RSE to do, and impractical before
we have a new RSE. (So maybe we're identifying another
characteristic of a good incoming RSE - be able to go find
out who reads RFCs and what they might want and bring that
info and/or proposals back to the putative "strategy WG" for
consideration.)

Cheers,
S.

---

From: Eric Rescorla <[email-masked]>
Date: Tue, 5 Jan 2021 13:43:13 -0800

> aren't represented in our discussions.)

Isn't this generally true? If you think of the IETF's ordinary output as
technologies as opposed to specifications, then it seems like the customers
of those technologies (e.g., end-users) basically aren't represented at
IETF. It's not clear to me how this is different. Indeed, arguably it's
better in that IETF participants generally are direct consumers of
technical specifications -- though of course not the only consumers -- so
hopefully have some idea of the concerns of other consumers.

-Ekr

---

From: Eliot Lear <[email-masked]>
Date: Wed, 6 Jan 2021 16:29:21 +0100

Top posting:

Can I just ask two questions?

As a practical matter, how has the “customer”/“reader” been represented in the past?
How should the “customer”/“reader” be represented in the future?

Is this something the RSE really did, and if so how?

Eliot

> On 5 Jan 2021, at 22:43, Eric Rescorla <[email-masked]> wrote:
> 
> 
> 
> On Tue, Jan 5, 2021 at 1:31 PM Brian E Carpenter <[email-masked] <mailto:[email-masked]>> wrote:
> Isn't this generally true? If you think of the IETF's ordinary output as technologies as opposed to specifications, then it seems like the customers of those technologies (e.g., end-users) basically aren't represented at IETF. It's not clear to me how this is different. Indeed, arguably it's better in that IETF participants generally are direct consumers of technical specifications -- though of course not the only consumers -- so hopefully have some idea of the concerns of other consumers.
> 
> -Ekr
> 
> 
>    Brian
> 
> > Cheers,
> > S.
> >
> [email-masked] <mailto:[email-masked]>
> https://www.iab.org/mailman/listinfo/rfced-future <https://www.iab.org/mailman/listinfo/rfced-future>
> --

---

From: John C Klensin <[email-masked]>
Date: Wed, 06 Jan 2021 12:15:32 -0500

Eliot,

I can't speak for the last administration or two (and hence am
not interpreting the "RSE" part of your questions narrowly) but,
historically, there has been keen awareness of the existence and
needs of those communities of "customers".  I even remember a
long-ago conversation from which I inferred that part of the
rationale for line-item RFC Editor funding was the needs of
procurement offices.  There was also keen awareness of a group
of customers/readers who have not been mentioned in the current
rounds of discussions: historians who might, 50, 100, or more
years from now, try to understand what had happened and why.

Because the "customer" may not actually read the RFCs and/or may
be a few steps removed (see Brian Rosen's explanation) and
because the "reader" may not present tense or even present
decade, perhaps you question should be about
"customer"/"reader"/"user".   I'm not picking nits here - as
soon as the question is about who is "represented" and how,
those differences become important.

Your question and that history brings us back to one of the
issues we keep circling around: If we want to be sure those
issues are addressed, one possibility is to write a job
description and then pick an RSE who is sensitive to the issues
and needs _and_ then give them authority.  If, instead, we are
determined to deal with those issues by community-populated
oversight bodies, then those bodies must be defined and
populated so they don't, collectively, lose sight of important
issues and thereby bring us to either a too-narrow focus on
easily measured objectives, notions that the RSE and/or the RPC
are "just contractors" who can be easily replaced, or both.  

best,
   john

--On Wednesday, January 6, 2021 16:29 +0100 Eliot Lear
<lear=[email-masked]> wrote:

> As a practical matter, how has the "customer"/"reader"
> been represented in the past? How should the
> "customer"/"reader" be represented in the future?
>

---

From: Eliot Lear <[email-masked]>
Date: Fri, 8 Jan 2021 15:56:23 +0100

Hi John,

Thanks for writing this.  I just have one follow-up point for you and the group.

> On 6 Jan 2021, at 18:15, John C Klensin <[email-masked]> wrote:
> 
> Eliot,
> 

There are several bars that this group can set to answer Russ’ question:

Better than what we have
What we have
Less than what we have

Of course we have to understand what we have to make a determination.  As to who performs this function, that could be the RSE or it could be others, or it could be no one, depending on where the group wants to set the bar.  This gets to a point that Mirja raised: it could be that we are overloading the function too much (I’m not advocating that view, I’m just stating that it has been mentioned in a broader context).

Eliot

---

From: John C Klensin <[email-masked]>
Date: Fri, 08 Jan 2021 13:58:55 -0500

--On Friday, January 8, 2021 15:56 +0100 Eliot Lear
<[email-masked]> wrote:

>> not interpreting the "RSE" part of your questions narrowly)
>> but, historically, there has been keen awareness of the
>> existence and needs of those communities of "customers".  I
>> even remember a long-ago conversation from which I inferred
>> that part of the rationale for line-item RFC Editor funding
>> was the needs of procurement offices.  There was also keen
>> awareness of a group of customers/readers who have not been
>> mentioned in the current rounds of discussions: historians
>> who might, 50, 100, or more years from now, try to understand
>> what had happened and why.
>...

> There are several bars that this group can set to answer
> Russ' question:
> 
> Better than what we have
> What we have
> Less than what we have

And we could try to set those bars differently depending on
whether "what we have" is interpreted as "today", "during
Heather's term before things started to melt down" [1], or
"during the Postel, or maybe Postel-Braden, period".   In the
second case, "before" because I think she became less effective
on a number of issues that are part of this discussion once
things started to go back and various parts of the
organizational structure started second-guessing her and/or
excluding her from relevant discussions.

In addition, one might answer differently depending on whether
"what we have" is interpreted in terms of general principles and
goals or in terms of particular sets of operational decisions
and procedures.  In particular, many of the principles (some
possibly not articulated explicitly and to a general audience)
that Postel tried to follow might still be applicable today but
things have changed a lot in 20+ years and we'd expect the
implementation details to be very different.  Some of the
principles have changed too: some of us can remember when there
was an assumption that the RFC Editor would perform a technical
review on most of the submitted documents.
 
> Of course we have to understand what we have to make a
> determination.  As to who performs this function, that could
> be the RSE or it could be others, or it could be no one,
> depending on where the group wants to set the bar.  This gets
> to a point that Mirja raised: it could be that we are
> overloading the function too much (I'm not advocating that
> view, I'm just stating that it has been mentioned in a
> broader context).

I see that concern as a question of trying to find the right
balance.  Certainly there is a risk of overloading the position
and, in particular, of heaping enough responsibilities and
requirements for different skills and expertise onto the job
that no one human, or at least no human we would have any hope
of recruiting, would be able to do it.  On the other hand, it is
not hard to imagine splitting functions up enough that no one
person really has a complete overview and priorities get set and
decisions made by whomever is most determined, loudest, and/or
has the most time on their hands to drag out arguments.  One
particular risk in that area is that, because almost any
decision might have financial implications, decisions could, by
default, end up being made on the basis of what is good for the
IETF Administration LLC rather than what is going for the
Internet or the RFC Series.
 
   john

[1] I'm not sure whether there is agreement on when the meltdown
started or if they might make a difference, but there are
clearly some plausible points prior to when Heather stepped down
or announced her intention to not seek a new contract.