Subject: [Suit] Alissa Cooper's Block on charter-ietf-suit-00-08: (with BLOCK and COMMENT)

From: Alissa Cooper <[email-masked]>
Date: Wed, 29 Nov 2017 11:21:51 -0800

Alissa Cooper has entered the following ballot position for
charter-ietf-suit-00-08: Block

When responding, please keep the subject line intact and reply to all
email addresses included in the To and CC lines. (Feel free to cut this
introductory paragraph, however.)

The document, along with other ballot positions, can be found here:
https://datatracker.ietf.org/doc/charter-ietf-suit/

----------------------------------------------------------------------
BLOCK:
----------------------------------------------------------------------

>From reading the mailing list and doing my own review, there seem to be a bunch
of open issues with this charter that would benefit from further discussion by
interested participants before the WG gets chartered. Some of these might just
be the result of lack of precision in the language used, but I think that is
actually pretty important for clarity in a WG charter. The issues are:

1. On the list there seems to be disagreement about the interpretation of this
text: "A lower number of formats is preferred to reduce code size for
supporting decoders on devices receiving a manifest and to maximize
interoperability of solutions." I also find myself confused by this (lower than
what? is there some upper bound?).

2. Michael Richardson raised some good question about this text that seem like
they warrant clarification, or at least a consensus call: "Software update
solutions that target updating software other than the firmware binaries are
also out of scope."

3. The milestones seem to use the term "Manifest format" to refer to something
that the charter calls "the contents of a mainfest" (I think), in contrast to
the multiple "formats" discussed in the charter. Given that there has also been
discussion on the list about format vs. serialization and the absence of a data
model specification, I think the charter and milestones would benefit from
being crystal clear about what deliverables the WG is expected to produce and
should use the same language throughout to name those deliverables.

4. This charter changed in some pretty important ways in the middle of the
external review period, but was never re-sent to the new-work mailing list.
There was also an error in the original announcement sent to new-work that
could cause confusion (it said this was a re-charter). I asked for it to be
re-sent but it doesn't look like it was. Since this is an important part of
external review, I really don't think this step should be skipped.

I'm willing to move to ABSTAIN if no one agrees with me but I thought I would
check to see if folks would be willing to take the time to sort out these
issues.

----------------------------------------------------------------------
COMMENT:
----------------------------------------------------------------------

Some lesser issues:

5. The charter now talks about how the group may describe the use of existing
discovery and transport mechanisms, but there are not associated milestone(s)
for this work. Not sure if this is intentional or an oversight.

6. I agree with the folks on the list who have said that the background
material about the IAB workshop and RFC 4108 should either be removed
(preferably), or shifted around to the beginning of the text and edited so that
it's clear that it's there for background purposes and not to constrain what
the group does going forward.

---

From: Dave Thaler <[email-masked]>
Date: Wed, 29 Nov 2017 19:29:43 +0000

My opinion inline below.

-----Original Message-----
From: Alissa Cooper [mailto:[email-masked]]
Sent: Wednesday, November 29, 2017 11:22 AM
To: The IESG <[email-masked]>
Cc: [email-masked]; [email-masked]
Subject: Alissa Cooper's Block on charter-ietf-suit-00-08: (with BLOCK and COMMENT)

When responding, please keep the subject line intact and reply to all email addresses included in the To and CC lines. (Feel free to cut this introductory paragraph, however.)

https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdatatracker.ietf.org%2Fdoc%2Fcharter-ietf-suit%2F&data=02%7C01%7Cdthaler%40microsoft.com%7C7cec1d4aab654d5176bd08d5375e727b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636475801151424497&sdata=oYrvLdWs6rbEAXVoyxqT3mbvSK71XDzJ%2BirdWWmamBM%3D&reserved=0

BLOCK:

>From reading the mailing list and doing my own review, there seem to be

>a bunch

of open issues with this charter that would benefit from further discussion by interested participants before the WG gets chartered. Some of these might just be the result of lack of precision in the language used, but I think that is actually pretty important for clarity in a WG charter. The issues are:

text: "A lower number of formats is preferred to reduce code size for supporting decoders on devices receiving a manifest and to maximize interoperability of solutions." I also find myself confused by this (lower than what? is there some upper bound?).

I believe the intent was just to keep it to some small number, but not necessarily 1 if the WG agrees that multiple are needed.   But we don’t want a proliferation of hundreds of them, or even really tens, since the presumed goal is for a management tool or service to support all formats needed to update a heterogeneous set of devices.

2. Michael Richardson raised some good question about this text that seem like they warrant clarification, or at least a consensus call: "Software update solutions that target updating software other than the firmware binaries are also out of scope."

That text (or at least the intent behind it) reflects the consensus during the SUIT BoF.   That is, software “used for boot” or some approximation thereof, and not “apps” that could be installed later on.

3. The milestones seem to use the term "Manifest format" to refer to something that the charter calls "the contents of a mainfest" (I think), in contrast to the multiple "formats" discussed in the charter. Given that there has also been discussion on the list about format vs. serialization and the absence of a data model specification, I think the charter and milestones would benefit from being crystal clear about what deliverables the WG is expected to produce and should use the same language throughout to name those deliverables.

I think the intent was that they were synonymous but yeah picking one would be less confusing.   Personally I think “manifest format(s)” is better than “the contents of a manifest”.

4. This charter changed in some pretty important ways in the middle of the external review period, but was never re-sent to the new-work mailing list.

There was also an error in the original announcement sent to new-work that could cause confusion (it said this was a re-charter). I asked for it to be re-sent but it doesn't look like it was. Since this is an important part of external review, I really don't think this step should be skipped.

I'm willing to move to ABSTAIN if no one agrees with me but I thought I would check to see if folks would be willing to take the time to sort out these issues.

COMMENT:

Some lesser issues:

5. The charter now talks about how the group may describe the use of existing discovery and transport mechanisms, but there are not associated milestone(s) for this work. Not sure if this is intentional or an oversight.

My opinion is that I consider it part of the architecture, but not sure what others thing.

6. I agree with the folks on the list who have said that the background material about the IAB workshop and RFC 4108 should either be removed (preferably), or shifted around to the beginning of the text and edited so that it's clear that it's there for background purposes and not to constrain what the group does going forward.

The consensus at the BoF was, from my recollection anyway, indeed to do so for RFC 4108.   I don’t recall BoF discussion about doing so for the IAB workshop too, but I am ok with that suggestion.

Dave

---

From: Kathleen Moriarty <[email-masked]>
Date: Wed, 29 Nov 2017 14:32:54 -0500

The chairs wanted to respond to the questions on list, so I will leave
that to them, but need to make an important correction below.

> https://datatracker.ietf.org/doc/charter-ietf-suit/

The charter did NOT change during the external review period.  The
charter was updated to version 7 prior to the start of the external
review, then the external review was requested.  The changes from
version 6 to 7 were a result of consensus calls during the BoF.  The
update to 8 was an attempt to fix a formatting issue, but the result
was no change. The BoF chairs were very clear with the consensus calls
in the room with the exact text being displayed and discussed.

The re-charter was likely because when the page was first created,
someone had put the charter text where charter text goes for an
established WG.

> check to see if folks would be willing to take the time to sort out these

The chairs will be responding on list and I said in an earlier email,
I would like to see the outcome before changing anything as I'd like
to follow our normal process and respect the chairs roll in the
process to assess consensus and drive the work forward.

Best regards,
Kathleen

-- 

Best regards,
Kathleen

---

From: Alissa Cooper <[email-masked]>
Date: Wed, 29 Nov 2017 15:17:08 -0500

> On Nov 29, 2017, at 2:32 PM, Kathleen Moriarty <[email-masked]> wrote:
>> 
> The charter did NOT change during the external review period.  

In the datatracker I see that the external review period began 11/3/17 with the -04 version of the charter and the message to new-work went out that day. https://datatracker.ietf.org/doc/charter-ietf-suit/history/ <https://datatracker.ietf.org/doc/charter-ietf-suit/history/>

Perhaps folks disagree about whether the changes are material, but the charter certainly changed from -04 to -07/-08.

Alissa

> The

---

From: Kathleen Moriarty <[email-masked]>
Date: Wed, 29 Nov 2017 15:27:23 -0500

On Wed, Nov 29, 2017 at 3:17 PM, Alissa Cooper <[email-masked]> wrote:
>
> On Nov 29, 2017, at 2:32 PM, Kathleen Moriarty
> <[email-masked]> wrote:
>
>
> In the datatracker I see that the external review period began 11/3/17 with
> the -04 version of the charter and the message to new-work went out that
> day. https://datatracker.ietf.org/doc/charter-ietf-suit/history/

Right, I forgot, however, the external review was set for an extended
period of time with the understanding that it could change from the
BoF, leaving 2 additional weeks for discussion after the changes were
made.

>
> Perhaps folks disagree about whether the changes are material, but the
> charter certainly changed from -04 to -07/-08.

The updates reflect consensus decisions of the BoF.  I was careful to
make sure they were posted 2 weeks prior to this telechat.

Regards,
Kathleen

-- 

Best regards,
Kathleen

---

From: Kathleen Moriarty <[email-masked]>
Date: Wed, 29 Nov 2017 15:33:52 -0500

> make sure they were posted 2 weeks prior to this telechat.

I do recall raising this point in one of the IESG/IAB joint
discussions on hot topics.

Regards,
Kathleen

-- 

Best regards,
Kathleen

---

From: Alissa Cooper <[email-masked]>
Date: Wed, 29 Nov 2017 15:51:07 -0500

> On Nov 29, 2017, at 3:27 PM, Kathleen Moriarty <[email-masked]> wrote:
> 
> On Wed, Nov 29, 2017 at 3:17 PM, Alissa Cooper <[email-masked] <mailto:[email-masked]>> wrote:
>> 

My point is that external review is, in part, targeted towards reviewers external to the IETF (i.e., folks participating in other SDOs). This is why we send proposed charters to new-work. Subscribers to that list saw an announcement of a charter (listed as a re-charter) on 11/3, asking for feedback by 11/13. Nothing further was sent to that list after the changes were made on 11/15, and there’s no reason why external reviewers subscribed to that list would know that the charter had changed or that the external review period was extended.

I think we need to act the way we would want other SDOs to act towards us.

> make sure they were posted 2 weeks prior to this telechat.

See above — they were posted to the datatracker, but not to new-work.

Alissa

---

From: Kathleen Moriarty <[email-masked]>
Date: Wed, 29 Nov 2017 16:07:43 -0500

On Wed, Nov 29, 2017 at 3:51 PM, Alissa Cooper <[email-masked]> wrote:
>
> On Nov 29, 2017, at 3:27 PM, Kathleen Moriarty
> My point is that external review is, in part, targeted towards reviewers
> external to the IETF (i.e., folks participating in other SDOs). This is why
> we send proposed charters to new-work. Subscribers to that list saw an
> announcement of a charter (listed as a re-charter) on 11/3, asking for
> feedback by 11/13. Nothing further was sent to that list after the changes
> were made on 11/15, and there’s no reason why external reviewers subscribed
> to that list would know that the charter had changed or that the external
> review period was extended.
>

I didn't think it was a big deal with the extended phase since the
IESG changes charters in external review regularly prior to final
publication.  The external review notification is a polite
communication and we sometimes skip that step with recharters and
often have text changes from list or IESG discussion prior to final
publication. This is the first time I've seen this called out to block
a charter progressing.  I had mentioned putting it in external review
early to dredge up external comments sooner and progress this faster
in case external entities were alerted to the BoF and to the charter
discussions.  The discussions all took place on list where there was
major text replacements.  There were other text updates that were no
accepted since there wasn't much agreement.  I haven't seen any
external responses from the review.  An email with an updated charter
should have gone out to your point, but it was posted 2 weeks prior to
the telechat.

> charter certainly changed from -04 to -07/-08.

This was based on discussions on list and supported by polls in the
BoF.  The last set of changes were made 2 weeks in advance of the
telechat.  This was mentioned in the joint IESG/IAB meetings.

> See above — they were posted to the datatracker, but not to new-work.

Noted, but that doesn't happen with other charters under review
either.  If you want to change process and tools, this shouldn't get
held up in that discussion.  The charter changes removed major
decisions from the charter to happen within the formed WG.  That alone
should alleviate concerns you might have.  If the chairs decide
additional changes are agreed, we'll update again as is normal in a
review phase.

Regards,
Kathleen

-- 

Best regards,
Kathleen

---

From: Alissa Cooper <[email-masked]>
Date: Wed, 29 Nov 2017 21:51:43 -0500

Since the IESG evaluation of this will be delayed, I think it would be beneficial to circulate an updated charter notification to new-work.

I think what is unusual here is running the external review process in parallel with the BoF process, rather than running the BoF first. I can’t recall another working group that has followed that path in recent memory.

Alissa

> On Nov 29, 2017, at 4:07 PM, Kathleen Moriarty <[email-masked]> wrote:
>

---

From: Kathleen Moriarty <[email-masked]>
Date: Wed, 29 Nov 2017 21:53:55 -0500

> I think what is unusual here is running the external review process in parallel with the BoF process, rather than running the BoF first. I can’t recall another working group that has followed that…

Sure, I thought it would be helpful to get the discussion all at once
rather than the external review possibly revisiting discussion points
that had been made.  The external review was set for a longer than
normal time period as well.  Since this work is bringing in new people
to the IETF and possibly interest from other SDOs, I thought this
would be helpful to further the work in a positive and collaborative
way.

Kathleen

-- 

Best regards,
Kathleen

---

From: Hannes Tschofenig <[email-masked]>
Date: Thu, 30 Nov 2017 09:21:18 +0000

My 5 cents.

Since the chartering work on this effort took so long there are obviously discussions ongoing already on the list that should actually better be done in a working group rather than on the charter itself.

I understand that you all want to have this charter to be as precise as possible. But, based on earlier discussions, we wanted to postpone some decisions to a later phase when those have been investigated in the group. For this reason I believe it is not possible to say how many formats we want to standardize other than stating the intention that the number ideally be small. The same is true for the actual data model/serialization format. Needless to say that different formats have different properties and offer more or less flexibility regarding the serialization format. Everyone seems to have a different preference for a format and the serialization. This is quite natural given that we are engineers and that's pretty much the only things we care about in life (besides the name of the group, of course).

Maybe we can, at this point in time, just say that the group will decide on the format. I am sure it is not the first time in the IETF history that there have been disagreements about formats. I am convinced that the IESG is actually very experienced in finding the most diplomatic wording for such a situation.

Since charters can be updated later I prefer to have the group figure out the technical pieces and that they are not captured in the charter yet. This should also remove the chicken-and-egg situation.

Ciao
Hannes

-----Original Message-----
From: Suit [mailto:[email-masked]] On Behalf Of Alissa Cooper
Sent: 29 November 2017 20:22
To: The IESG
Cc: [email-masked]; [email-masked]
Subject: [Suit] Alissa Cooper's Block on charter-ietf-suit-00-08: (with BLOCK and COMMENT)

Some lesser issues:

_______________________________________________
Suit mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/suit
IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.

---

From: Michael Richardson <[email-masked]>
Date: Thu, 30 Nov 2017 11:50:04 -0500

Hannes Tschofenig <[email-masked]> wrote:
    > I understand that you all want to have this charter to be as precise as
    > possible. But, based on earlier discussions, we wanted to postpone some
    > decisions to a later phase when those have been investigated in the
    > group. For this reason I believe it is not possible to say how many
    > formats we want to standardize other than stating the intention that
    > the number ideally be small. The same is true for the actual data
    > model/serialization format. Needless to say that different formats have
    > different properties and offer more or less flexibility regarding the
    > serialization format. Everyone seems to have a different preference for
    > a format and the serialization. This is quite natural given that we are
    > engineers and that's pretty much the only things we care about in life
    > (besides the name of the group, of course).

+1

    > Since charters can be updated later I prefer to have the group figure
    > out the technical pieces and that they are not captured in the charter
    > yet. This should also remove the chicken-and-egg situation.

Agreed.

    alissa> 2. Michael Richardson raised some good question about this text
    alissa> that seem like they warrant clarification, or at least a
    alissa> consensus call: "Software update solutions that target updating
    alissa> software other than the  firmware binaries are also out of scope."

BTW, this would not be a blocking comment from me.
I can live with the text as is, I'd just prefer clarification.

--
Michael Richardson <[email-masked]>, Sandelman Software Works
 -= IPv6 IoT consulting =-