Subject: Re: [RTG-DIR] RTG-DIR last call review of draft-ietf-detnet-data-plane-framework-03

From: Balázs Varga A <[email-masked]>
Date: Tue, 7 Jan 2020 14:32:14 +0000

Hi Sasha,

Thanks for the review comments/nits.

Some of them require some discussion with co-authors,
so I will be back with a structured response soon.

Many thanks
Bala’zs

From: Alexander Vainshtein <[email-masked]>
Sent: Thursday, December 26, 2019 5:30 PM
To: [email-masked]
Cc: [email-masked]; [email-masked]; [email-masked]; Yemin (Amy) <[email-masked]>
Subject: RTG-DIR last call review of draft-ietf-detnet-data-plane-framework-03

Hello,

I have been selected as the Routing Directorate reviewer for this draft. The Routing Directorate seeks to review all routing or routing-related drafts as they pass through IETF last call and IESG review, and sometimes on special request. The purpose of the review is to provide assistance to the Routing ADs. For more information about the Routing Directorate, please see ​http://trac.tools.ietf.org/area/rtg/trac/wiki/RtgDir<https://clicktime.symantec.com/3HoggwpVJ3tAkafdde3M5kS6H2?u=http%3A%2F%2Ftrac.tools.ietf.org%2Farea%2Frtg%2Ftrac%2Fwiki%2FRtgDir>

Although these comments are primarily for the use of the Routing ADs, it would be helpful if you could consider them along with any other IETF Last Call comments that you receive, and strive to resolve them through discussion or by updating the draft.

Document: draft-ietf-detnet-data-plane-framework-03.txt
Reviewer: Alexander (“Sasha”) Vainshtein
Review Date: 26-Dec-19
IETF LC End Date: Not known
Intended Status: Informational

Summary:
I have some minor concerns about this document that I think should be resolved before publication.

Comments:
The draft is one of a group of DetNet documents.
One of these documents has been already published as RFC 8655<https://clicktime.symantec.com/3G9XdBmhXdcKZ9XbdXGkvRB6H2?u=https%3A%2F%2Ftools.ietf.org%2Fhtml%2Frfc8655>, while several others are in different stages of the IETF process.
These documents seem to be closely related, and this makes reading and understanding the DetNet Data Plane Framework draft complicated (at least for somebody, like me, who is not deeply immersed in the topic).
Specifically, reading RFC 8655 seems to me a mandatory prerequisite for understanding the data plane framework draft.

I would also like to notice that there are 7 authors listed on the front page of  the draft.
I defer to the WG chairs and the ADs to decide whether this is acceptable, or any changes are required.

I have privately discussed my original comments with the authors prior to posting this review, and received highly relevant feedback that has helped me to resolve some of my concerns and to clarify some of the remaining ones.  I also believe that we have reached a rough consensus regarding disposition of the majority of my comments. I would like to express my gratitude to the authors of the draft for their  responsiveness and cooperation.

Major Issues: No major issues found.

Minor Issues:

  1.  As mentioned above, RFC 8655 looks to me a mandatory pre-requisite for reading and understanding this draft. Therefore I suggest making it a Normative reference (currently there is just an Informative reference to the draft that already has been published as RFC 8655). This issue has been discussed with the authors, and, as I can see it, there were no objections to such a change.
  2.  After reading both  RFC 8655 and the DetNet Data Plane Framework draft I have failed to understand whether the “Packet Ordering Function (POF)” is expected to actually reorder (or try to reorder) packets that have been received out of order, or could simply discard such packets:
     *   On one hand:

                                                  i.     The DetNet Data Plane Framework draft says in Section one that “The service sub-layer is used to provide  DetNet service protection and reordering”

                                                 ii.     Section 3.2.2.2 of RFC 8655 says that “The POF uses the sequencing information to reorder a DetNet flow's  packets that are received out of order”

     *   On the other hand, neither of these documents mentions the need for additional resources (buffers and timers) that are required for reordering of packets received out of order, and impact of this operation on the packet delay variation (a.k.a. jitter). What’s more, allocation of these resources (if they are used) would have to be done at the DetNet service sub-layer, and this seems to contradict RFC 8655  where allocation of resources is associated just with the forwarding sub-layer (see Figure 2 in Section 4.1.1 of RFC 8655 that is also reproduced verbatim in the DetNet Data Plane Framework draft).
     *   For comparison, the PWE3 documents that deal with sequencing and reordering clearly differentiate between reordering and discard of packets that have been received out of order:

                                                  i.     Section 4.2 of RFC 4385<https://tools.ietf.org/html/rfc4385> provides a clear definition of PWE packets received in and  of order. It then says that “If the packet is found to be in order, it MAY be delivered  immediately” and “Packets that are received out of order MAY either be dropped or  reordered.  The choice between dropping or reordering an out-of-sequence packet is at the discretion of the receiver”.  I.e., ordering can be achieved by simply dropping packets that have been received out of order

                                                 ii.     Section 7.3.2 of RFC 4197<https://tools.ietf.org/html/rfc4179> (that deals with TDM PWs) says that  packets received out of order “SHOULD be reordered if not judged to be too late or too early for  playout”

     *   From my POV the DetNet Data Plane Framework draft should clearly define the expectations from the POF regarding handling of packets that have been received out-of-order, and the impact of reordering (if it is used) on the goals of the DetNet services.
  1.  Elimination of replicated packets is yet another important function of the DetNet data plane that seems to be under-specified. From my POV ability to perform this function depends on the (worst case of the) differential delay between the paths taken through the network by the multiple copies of the packet, but I have not found any discussion of such a linkage in the draft.
  2.  The DetNet Data Plane Framework draft mentions in the Introduction that  TSN as a possible (but not mandatory) underlay for carrying the DetNet flows, and says that “some of the DetNet benefits can be gained by running over a data link layer that has not been specifically enhanced to support TSN”. It would be really nice if the draft would also specify which (if any) of the DetNet goals could not be achieved if it runs over the data link layer that has not been enhanced to support TSN. Specifically, I would like to see:
     *   Whether the DetNet data plane is expected to provide any equivalent of the frame preemption function defined in IEEE802.1Qbu

                                                  i.     The relevant text in RFC 8655 only says that all the TSN techniques are currently defined for Ethernet and Layer 2 bridging, “they are all, except perhaps for packet preemption, equally applicable to media other than Ethernet and to routers as well as bridges”

                                                 ii.     Personally I believe that preemption is closely related to usage of specific media, but this may be due to my admitted ignorance in these matters

     *   What could be the impact of NOT supporting frame preemption in DetNet on its goals, especially on jitter?
  1.  The draft states, in Section 4.2.4., that “DetNet applications typically generate bidirectional traffic”.
     *   I have looked up RFC 8557<https://tools.ietf.org/html/rfc8557> and, especially, RFC 8578<https://tools.ietf.org/html/rfc8578>, but did not find there any justifications for this statement. In fact, some of the application listed in RFC 8578 (e.g. all applications related to audio and video) seem to be inherently unidirectional.
     *   I think that some clarification, preferably with references to specific DetNet use cases that require bidirectional traffic, would be useful
  2.  The term “d-CW” appears several times in the draft without either am explicit definition or a clear reference to such a definition.
     *   The definition of d-CW seems to be in Section 4.2.1 of the DetNet Data Plane: MPLS<https://tools.ietf.org/html/draft-ietf-detnet-mpls-04> draft

                                                  i.     If this is correct, then an explicit reference to this draft should be added IMHO

                                                 ii.     I am not sure if this would make  the DetNet Data Plane: MPLS draft a Normative reference to this draft, or it can be left as an Informative reference (as today)

     *   I find it somewhat surprising that the  term d-CW d is not used in Section 3.5. “DetNet MPLS Data Plane” of the DetNet Data Plane Framework draft. Instead, it only mentions “a shim layer called a control word (CW)” followed by a reference to RFC 4385.
  1.  In Section 4 the draft describes the DetNet data plane requirements to the DetNet Controller Plane without providing any explicit definition or a reference of the latter:
     *   It seems that the relevant definition appears in Section 4.4.2 of RFC 8655
     *   IMHO adding this reference explicitly (or even reproducing it verbatim) would help the reader to understand why, say, allocation and distribution of S-labels and F-labels are required from the DetNet Controller plane in the draft
  2.  Section 3.6.1.4 “Network Coding” says that “Network Coding, not to be confused with network programming, comprises several techniques where multiple data flows are encoded”.
     *   No further explanation or reference is provided
     *   I freely admit complete ignorance in all matters dealing with Network Coding, therefore I did not understand why this section appears in the draft.
     *   One of the authors has pointed to the IRTF Coding for efficient NetWork Communications<hhttps://datatracker.ietf.org/rg/nwcrg/about/> Research Group (NWCRG)  as the possible source of information about Network Coding, and this was really helpful to me. I think that adding at least this pointer (and possibly some others) as Informative Reference(s) to the draft,  would be helpful  to other readers as well.

Nits:
I did not run the nits check on the draft. So I just included two sentences that looked problematic to me. However, I am not a native English speaker, so please consider these with a grain of salt.

  1.  Something seems to be wrong with the grammar in the following sentence in Section 3.6.1.1: “Misbehaving DetNet flows must be detected and it have to be ensured that they do not compromise QoS of other flows”
  2.  The next sentence in the same section “The use of (queuing, policing, shaping) policies can be used to ensure that the allocation of resources reserved for DetNet is met” IMHO should be rephrased e.g., like “Queuing, policing, and shaping policies can be used to ensure that the allocation of resources reserved for DetNet is met”.

Hopefully, these notes will be useful.

Regards,
Sasha

Office: +972-39266302
Cell:      +972-549266302
Email:   [email-masked]<mailto:[email-masked]>

___________________________________________________________________________

This e-mail message is intended for the recipient only and contains information which is
CONFIDENTIAL and which may be proprietary to ECI Telecom. If you have received this
transmission in error, please inform us by e-mail, phone or fax, and then delete the original
and all copies thereof.
___________________________________________________________________________

---

From: Alexander Vainshtein <[email-masked]>
Date: Tue, 7 Jan 2020 16:05:39 +0000

Balazs,
No problem, I"ll be waiting patiently.

Regards and best (if belated) New Uear wishes,
Sasha
Get Outlook for Android<https://aka.ms/ghei36>

________________________________
From: Balázs Varga A <[email-masked]>
Sent: Tuesday, January 7, 2020, 16:32
To: Alexander Vainshtein; [email-masked]
Cc: [email-masked]; [email-masked]; [email-masked]; Yemin (Amy)
Subject: RE: RTG-DIR last call review of draft-ietf-detnet-data-plane-framework-03

Hi Sasha,

Many thanks
Bala’zs

Hello,

Summary:

Comments:

Minor Issues:

                                                  i.     Section 4.2 of RFC 4385<https://clicktime.symantec.com/33EPoJPGK3P16uEhFVxEQPF6H2?u=https%3A%2F%2Ftools.ietf.org%2Fhtml%2Frfc4385> provides a clear definition of PWE packets received in and  of order. It then says that “If the packet is found to be in order, it MAY be delivered  immediately” and “Packets that are received out of order MAY either be dropped or  reordered.  The choice between dropping or reordering an out-of-sequence packet is at the discretion of the receiver”.  I.e., ordering can be achieved by simply dropping packets that have been received out of order

                                                 ii.     Section 7.3.2 of RFC 4197<https://clicktime.symantec.com/37VT1KAX7zdhNmeuVcyLo5c6H2?u=https%3A%2F%2Ftools.ietf.org%2Fhtml%2Frfc4179> (that deals with TDM PWs) says that  packets received out of order “SHOULD be reordered if not judged to be too late or too early for  playout”

     *   What could be the impact of NOT supporting frame preemption in DetNet on its goals, especially on jitter?
  1.  The draft states, in Section 4.2.4., that “DetNet applications typically generate bidirectional traffic”.
     *   I have looked up RFC 8557<https://clicktime.symantec.com/3S5RkymWb9AHCmyh98pDnNf6H2?u=https%3A%2F%2Ftools.ietf.org%2Fhtml%2Frfc8557> and, especially, RFC 8578<https://clicktime.symantec.com/32QD67HM2j2fhREf17yZBwp6H2?u=https%3A%2F%2Ftools.ietf.org%2Fhtml%2Frfc8578>, but did not find there any justifications for this statement. In fact, some of the application listed in RFC 8578 (e.g. all applications related to audio and video) seem to be inherently unidirectional.
     *   I think that some clarification, preferably with references to specific DetNet use cases that require bidirectional traffic, would be useful
  2.  The term “d-CW” appears several times in the draft without either am explicit definition or a clear reference to such a definition.
     *   The definition of d-CW seems to be in Section 4.2.1 of the DetNet Data Plane: MPLS<https://clicktime.symantec.com/3saUg5oC3fvZTGYHgpTVTz6H2?u=https%3A%2F%2Ftools.ietf.org%2Fhtml%2Fdraft-ietf-detnet-mpls-04> draft

     *   One of the authors has pointed to the IRTF Coding for efficient NetWork Communications Research Group (NWCRG)  as the possible source of information about Network Coding, and this was really helpful to me. I think that adding at least this pointer (and possibly some others) as Informative Reference(s) to the draft,  would be helpful  to other readers as well.

Nits:

Regards,
Sasha

Office: +972-39266302
Cell:      +972-549266302

---

From: "Don Fedyk" <[email-masked]>
Date: Mon, 13 Jan 2020 17:39:32 -0500

Hi Sasha 

Thank you for your thorough review. I have attempted to answer your

comments one by one see[Don] with old/new.  If there are additional changes
on your items it might

be good just to break off any item in a separate follow-up. I have included
a

full side by side diff so you can examine the changes in place. (hopefully
this will make through the mail filters).

If not I can send the complete proposed updated draft text – just let us
know.  

Cheers

Don

From: Alexander Vainshtein <[email-masked]
<mailto:[email-masked]> > 
Sent: Thursday, December 26, 2019 5:30 PM
To: [email-masked] <mailto:[email-masked]> 
Cc: [email-masked] <mailto:[email-masked]> ; [email-masked]
<mailto:[email-masked]> ;
[email-masked]
<mailto:[email-masked]> ; Yemin
(Amy) <[email-masked] <mailto:[email-masked]> >
Subject: RTG-DIR last call review of
draft-ietf-detnet-data-plane-framework-03

Hello, 

I have been selected as the Routing Directorate reviewer for this draft.
The Routing Directorate seeks to review all routing or routing-related
drafts as they pass through IETF last call and IESG review, and sometimes on
special request. The purpose of the review is to provide assistance to the
Routing ADs. For more information about the Routing Directorate, please see
<https://clicktime.symantec.com/3HoggwpVJ3tAkafdde3M5kS6H2?u=http%3A%2F%2Ftr
ac.tools.ietf.org%2Farea%2Frtg%2Ftrac%2Fwiki%2FRtgDir>
​http://trac.tools.ietf.org/area/rtg/trac/wiki/RtgDir 

Although these comments are primarily for the use of the Routing ADs, it
would be helpful if you could consider them along with any other IETF Last
Call comments that you receive, and strive to resolve them through
discussion or by updating the draft. 

Summary: 

I have some minor concerns about this document that I think should be
resolved before publication.

Comments:

One of these documents has been already published as RFC 8655
<https://clicktime.symantec.com/3G9XdBmhXdcKZ9XbdXGkvRB6H2?u=https%3A%2F%2Ft
ools.ietf.org%2Fhtml%2Frfc8655> , while several others are in different
stages of the IETF process. 

These documents seem to be closely related, and this makes reading and
understanding the DetNet Data Plane Framework draft complicated (at least
for somebody, like me, who is not deeply immersed in the topic).

Specifically, reading RFC 8655 seems to me a mandatory prerequisite for
understanding the data plane framework draft.

I would also like to notice that there are 7 authors listed on the front
page of  the draft. 

I defer to the WG chairs and the ADs to decide whether this is acceptable,
or any changes are required.

 [Don] Two authors are now contributors. 

Section 7., paragraph 3:

OLD:

8.1.  Normative References

NEW:

    The following people contributed substantially to the content of this

    document:

       Don Fedyk

       Jouni Korhonen

I have privately discussed my original comments with the authors prior to
posting this review, and received highly relevant feedback that has helped
me to resolve some of my concerns and to clarify some of the remaining ones.
I also believe that we have reached a rough consensus regarding disposition
of the majority of my comments. I would like to express my gratitude to the
authors of the draft for their  responsiveness and cooperation.

Minor Issues: 

1.	As mentioned above, RFC 8655 looks to me a mandatory pre-requisite
for reading and understanding this draft. Therefore I suggest making it a
Normative reference (currently there is just an Informative reference to the
draft that already has been published as RFC 8655). This issue has been
discussed with the authors, and, as I can see it, there were no objections
to such a change.

[Don]Normative Reference. Done

OLD:

8.2.  Informative References

NEW:

    [RFC8655]  Finn, N., Thubert, P., Varga, B., and J. Farkas,

               "Deterministic Networking Architecture", RFC 8655,

               DOI 10.17487/RFC8655, October 2019,

               <https://www.rfc-editor.org/info/rfc8655>.

2.	After reading both  RFC 8655 and the DetNet Data Plane Framework
draft I have failed to understand whether the “Packet Ordering Function
(POF)” is expected to actually reorder (or try to reorder) packets that have
been received out of order, or could simply discard such packets:

a.	On one hand:

                                                  i.     The DetNet Data
Plane Framework draft says in Section one that “The service sub-layer is
used to provide  DetNet service protection and reordering”

                                                 ii.     Section 3.2.2.2 of
RFC 8655 says that “The POF uses the sequencing information to reorder a
DetNet flow's  packets that are received out of order”

b.	On the other hand, neither of these documents mentions the need for
additional resources (buffers and timers) that are required for reordering
of packets received out of order, and impact of this operation on the packet
delay variation (a.k.a. jitter). What’s more, allocation of these resources
(if they are used) would have to be done at the DetNet service sub-layer,
and this seems to contradict RFC 8655  where allocation of resources is
associated just with the forwarding sub-layer (see Figure 2 in Section 4.1.1
of RFC 8655 that is also reproduced verbatim in the DetNet Data Plane
Framework draft).
c.	For comparison, the PWE3 documents that deal with sequencing and
reordering clearly differentiate between reordering and discard of packets
that have been received out of order:

[Don] Ordering is at the server sub-layer. Resources - buffers for
reordering are at this layer. Updated

Section 1., paragraph 5:

OLD:

    DetNet flows may be carried over network technologies that can

    provide the DetNet required service characteristics.  For example,

    DetNet MPLS flows can be carried over IEEE 802.1 Time Sensitive

    Network (TSN) [IEEE802.1TSNTG] sub-networks.  However, IEEE 802.1 TSN

    support is not required and some of the DetNet benefits can be gained

    by running over a data link layer that has not been specifically

    enhanced to support TSN.

NEW:

    DetNet flows may be carried over network technologies that can

    provide the DetNet required service characteristics.  For example,

    DetNet MPLS flows can be carried over IEEE 802.1 Time Sensitive

    Network (TSN) [IEEE802.1TSNTG] sub-networks.  However, IEEE 802.1 TSN

    support is not required in DetNet.  TSN frame preemption is an

    example of a forwarding layer capability that is typically not

    replicated in other forwarding technologies.  Most of DetNet benefits

    can be gained by running over a data link layer that has not been

    specifically enhanced to support all TSN capabilities but for certain

    networks and traffic mixes delay and jitter performance may vary due

    to the forwarding sub-layer intrinsic properties.

Section 4.2.3., paragraph 1:

OLD:

    As discussed in Section 1, this document does not specify the

    mechanisms needed to eliminate packet contention, packet loss or

    reduce jitter for DetNet flows at the DetNet forwarding sub-layer.

    The ability to manage node and link resources to be able to provide

    these functions is a necessary part of the DetNet controller plane.

    It is also necessary to be able to control the required queuing

    mechanisms used to provide these functions along a flow's path

    through the network.  See [I-D.ietf-detnet-ip] and Section 4.1 for

    further discussion of these requirements.

NEW:

    As discussed in Section 1, this document does not specify the

    mechanisms needed to eliminate packet contention, packet loss or

    reduce jitter for DetNet flows at the DetNet forwarding sub-layer.

    The ability to manage node and link resources to be able to provide

    these functions is a necessary part of the DetNet controller plane.

    It is also necessary to be able to control the required queuing

    mechanisms used to provide these functions along a flow's path

    through the network.  See [I-D.ietf-detnet-ip] and Section 4.1 for

    further discussion of these requirements.  Some forms of protection

    may enforce packet loss or change jitter characteristics in the cases

    where packets are reordered when out-of-order packets are received at

    the service sub-layer.

                                                  i.     Section 4.2 of RFC
4385
<https://clicktime.symantec.com/33EPoJPGK3P16uEhFVxEQPF6H2?u=https%3A%2F%2Ft
ools.ietf.org%2Fhtml%2Frfc4385>  provides a clear definition of PWE packets
received in and  of order. It then says that “If the packet is found to be
in order, it MAY be delivered  immediately” and “Packets that are received
out of order MAY either be dropped or  reordered.  The choice between
dropping or reordering an out-of-sequence packet is at the discretion of the
receiver”.  I.e., ordering can be achieved by simply dropping packets that
have been received out of order

                                                 ii.     Section 7.3.2 of
RFC 4197
<https://clicktime.symantec.com/37VT1KAX7zdhNmeuVcyLo5c6H2?u=https%3A%2F%2Ft
ools.ietf.org%2Fhtml%2Frfc4179>  (that deals with TDM PWs) says that
packets received out of order “SHOULD be reordered if not judged to be too
late or too early for  playout”

d.	From my POV the DetNet Data Plane Framework draft should clearly
define the expectations from the POF regarding handling of packets that have
been received out-of-order, and the impact of reordering (if it is used) on
the goals of the DetNet services.

[Don] add a text to clarify. (also above change).

Section 3., paragraph 7:

OLD:

    The service sub-layer provides additional support beyond the

    connectivity function of the forwarding sub-layer.  An example of

    this is Packet Replication, Elimination, and Ordering functions see

    Section 4.3.

NEW:

    The service sub-layer provides additional support beyond the

    connectivity function of the forwarding sub-layer.  An example of

    this is Packet Replication, Elimination, and Ordering functions see

    Section 4.3.  The ordering (POF) uses sequence numbers added to

    packets enabling a range of packet order protection from simple

    ordering and dropping out-of-order packets to more complex reordering

    of a fixed number of out-of-order, minimally delayed packets.

    Reordering requires buffer resources and has impact on the delay and

    jitter of packets in the DetNet flow.

3.	Elimination of replicated packets is yet another important function
of the DetNet data plane that seems to be under-specified. From my POV
ability to perform this function depends on the (worst case of the)
differential delay between the paths taken through the network by the
multiple copies of the packet, but I have not found any discussion of such a
linkage in the draft.

[Don] See above change. And new section.

OLD:

3.6.2.2.  Ring Service Protection

NEW:

3.6.2.2.  Path Differential Delay

    In the preceding example, reordering of packets is dependent on the

    number of out-of-order packets that can be buffered and the delay

    difference of arriving packets.  DetNet uses configuration of the

    number of maximum number out-of-order packets and the maximum delay

    that out-of-order packets can be held before being delivered.  If the

    differential delay between paths is large enough or the arrival rate

    of packets large enough packets may be dropped instead of being

    reordered.  Likewise elimination uses the sequence number to

    eliminate duplicate packets and large differential delays combined

    with high numbers of packets may exceed the ability of the

    elimination function to eliminate all duplicate packets.  Network

    engineering of DetNet services needs to take these factors into

    account.

 3.6.2.3.  Ring Service Protection

4.	The DetNet Data Plane Framework draft mentions in the Introduction
that  TSN as a possible (but not mandatory) underlay for carrying the DetNet
flows, and says that “some of the DetNet benefits can be gained by running
over a data link layer that has not been specifically enhanced to support
TSN”. It would be really nice if the draft would also specify which (if any)
of the DetNet goals could not be achieved if it runs over the data link
layer that has not been enhanced to support TSN. Specifically, I would like
to see:

1.	Whether the DetNet data plane is expected to provide any equivalent
of the frame preemption function defined in IEEE802.1Qbu

[Don] Clarified 

Section 1., paragraph 3:

OLD:

    The DetNet Architecture models the DetNet related data plane

    functions decomposed into two sub-layers: a service sub-layer and a

    forwarding sub-layer.  The service sub-layer is used to provide

    DetNet service protection and reordering.  The forwarding sub-layer

    leverages Traffic Engineering mechanisms and provides congestion

    protection (low loss, assured latency, and limited out-of-order

    delivery).

NEW:

    The DetNet Architecture models the DetNet related data plane

    functions decomposed into two sub-layers: a service sub-layer and a

    forwarding sub-layer.  The service sub-layer is used to provide

    DetNet service protection and reordering.  The forwarding sub-layer

    leverages Traffic Engineering mechanisms and provides congestion

    protection (low loss, assured latency, and limited out-of-order

    delivery).  A particular forwarding sub-layer may have capabilities

    that are not available on other forwarding-sub layers.  DetNet makes

    use of the existing forwarding sub-layers with their respective

    capabilities and does not require 1:1 equivalence between different

    forwarding sub-layer capabilities.

Section 4.2.3., paragraph 1:

OLD:

    As discussed in Section 1, this document does not specify the

    mechanisms needed to eliminate packet contention, packet loss or

    reduce jitter for DetNet flows at the DetNet forwarding sub-layer.

    The ability to manage node and link resources to be able to provide

    these functions is a necessary part of the DetNet controller plane.

    It is also necessary to be able to control the required queuing

    mechanisms used to provide these functions along a flow's path

    through the network.  See [I-D.ietf-detnet-ip] and Section 4.1 for

    further discussion of these requirements.

NEW:

    As discussed in Section 1, this document does not specify the

    mechanisms needed to eliminate packet contention, packet loss or

    reduce jitter for DetNet flows at the DetNet forwarding sub-layer.

    The ability to manage node and link resources to be able to provide

    these functions is a necessary part of the DetNet controller plane.

    It is also necessary to be able to control the required queuing

    mechanisms used to provide these functions along a flow's path

    through the network.  See [I-D.ietf-detnet-ip] and Section 4.1 for

    further discussion of these requirements.  Some forms of protection

    may enforce packet loss or change jitter characteristics in the cases

    where packets are reordered when out-of-order packets are received at

    the service sub-layer.

                                                  i.     The relevant text
in RFC 8655 only says that all the TSN techniques are currently defined for
Ethernet and Layer 2 bridging, “they are all, except perhaps for packet
preemption, equally applicable to media other than Ethernet and to routers
as well as bridges”

                                                 ii.     Personally I
believe that preemption is closely related to usage of specific media, but
this may be due to my admitted ignorance in these matters

b.	What could be the impact of NOT supporting frame preemption in
DetNet on its goals, especially on jitter?

[Don] Hopefully above text changes cover this.

5.	The draft states, in Section 4.2.4., that “DetNet applications
typically generate bidirectional traffic”.  

1.	I have looked up RFC 8557
<https://clicktime.symantec.com/3S5RkymWb9AHCmyh98pDnNf6H2?u=https%3A%2F%2Ft
ools.ietf.org%2Fhtml%2Frfc8557>  and, especially, RFC 8578
<https://clicktime.symantec.com/32QD67HM2j2fhREf17yZBwp6H2?u=https%3A%2F%2Ft
ools.ietf.org%2Fhtml%2Frfc8578> , but did not find there any justifications
for this statement. In fact, some of the application listed in RFC 8578
(e.g. all applications related to audio and video) seem to be inherently
unidirectional. 
2.	I think that some clarification, preferably with references to
specific DetNet use cases that require bidirectional traffic, would be
useful

[Don] 

Section 4.2.4., paragraph 1:

OLD:

    DetNet applications typically generate bidirectional traffic.  IP and

    MPLS typically treat each direction separately and do not force

    interdependence of each direction.  MPLS has considered bidirectional

    traffic requirements and the MPLS definitions from [RFC5654] are

    useful to illustrate terms such as associated bidirectional flows and

    co-routed bidirectional flows.  MPLS defines a point-to-point

    associated bidirectional LSP as consisting of two unidirectional

    point-to-point LSPs, one from A to B and the other from B to A, which

    are regarded as providing a single logical bidirectional forwarding

    path.  This is analogous to standard IP routing.  MPLS defines a

    point-to-point co-routed bidirectional LSP as an associated

    bidirectional LSP which satisfies the additional constraint that its

    two unidirectional component LSPs follow the same path (in terms of

    both nodes and links) in both directions.  An important property of

    co-routed bidirectional LSPs is that their unidirectional component

    LSPs share fate.  In both types of bidirectional LSPs, resource

    reservations may differ in each direction.  The concepts of

    associated bidirectional flows and co-routed bidirectional flows can

    also be applied to DetNet IP flows.

NEW:

    In many cases DetNet flows can be considered unidirectional and

    independent.  However, there are cases where the DetNet service

    requires bidirectional traffic from a DetNet application service

    perspective.  IP and MPLS typically treat each direction separately

   and do not force interdependence of each direction.  MPLS has

    considered bidirectional traffic requirements and the MPLS

    definitions from [RFC5654] are useful to illustrate terms such as

    associated bidirectional flows and co-routed bidirectional flows.

    MPLS defines a point-to-point associated bidirectional LSP as

    consisting of two unidirectional point-to-point LSPs, one from A to B

    and the other from B to A, which are regarded as providing a single

    logical bidirectional forwarding path.  This is analogous to standard

    IP routing.  MPLS defines a point-to-point co-routed bidirectional

    LSP as an associated bidirectional LSP which satisfies the additional

    constraint that its two unidirectional component LSPs follow the same

    path (in terms of both nodes and links) in both directions.  An

    important property of co-routed bidirectional LSPs is that their

    unidirectional component LSPs share fate.  In both types of

    bidirectional LSPs, resource reservations may differ in each

    direction.  The concepts of associated bidirectional flows and co-

    routed bidirectional flows can also be applied to DetNet IP flows.

5.	The term “d-CW” appears several times in the draft without either am
explicit definition or a clear reference to such a definition. 

1.	The definition of d-CW seems to be in Section 4.2.1 of the DetNet
Data Plane: MPLS
<https://clicktime.symantec.com/3saUg5oC3fvZTGYHgpTVTz6H2?u=https%3A%2F%2Fto
ols.ietf.org%2Fhtml%2Fdraft-ietf-detnet-mpls-04>  draft

[Don] Definitions 

    CW            Control Word.

    d-CW          DetNet Control Word.

i.	If this is correct, then an explicit reference to this draft should
be added IMHO

[Don] will add reference. 

Section 3.1.2., paragraph 1:

OLD:

    DetNet encodes specific flow attributes (flow identity and sequence

    number) in packets.  For example, in DetNet IP, zero encapsulation is

    used and no sequence number is available, and in DetNet MPLS, DetNet

    specific information may be added explicitly to the packets in the

    format of S-label and d-CW.

NEW:

    DetNet encodes specific flow attributes (flow identity and sequence

    number) in packets.  For example, in DetNet IP, zero encapsulation is

    used and no sequence number is available, and in DetNet MPLS, DetNet

    specific information may be added explicitly to the packets in the

    format of S-label and d-CW [I-D.ietf-detnet-mpls] .

Section 3.5., paragraph 3:

OLD:

    In cases where metadata is needed to process an MPLS encapsulated

    packet at the service sub-layer, a shim layer called a control word

    (CW) [RFC4385] can be used.  Although such CWs are frequently 32 bits

    long, there is no architectural constraint on its size of this

    structure, only the requirement that it is fully understood by all

    parties operating on it in the DetNet service sub-layer.  The

    operation of this method is described in detail in

    [I-D.ietf-detnet-mpls].

NEW:

    In cases where metadata is needed to process an MPLS encapsulated

    packet at the service sub-layer, the d-CW [I-D.ietf-detnet-mpls],

    [RFC4385],can be used.  Although such d-CWs are frequently 32 bits

    long, there is no architectural constraint on its size of this

    structure, only the requirement that it is fully understood by all

    parties operating on it in the DetNet service sub-layer.  The

    operation of this method is described in detail in

    [I-D.ietf-detnet-mpls].

                                                 ii.     I am not sure if
this would make  the DetNet Data Plane: MPLS draft a Normative reference to
this draft, or it can be left as an Informative reference (as today)

[Don] Left as informative.

b.	I find it somewhat surprising that the  term d-CW d is not used in
Section 3.5. “DetNet MPLS Data Plane” of the DetNet Data Plane Framework
draft. Instead, it only mentions “a shim layer called a control word (CW)”
followed by a reference to RFC 4385.  

[Don] See above changes.

7.	In Section 4 the draft describes the DetNet data plane requirements
to the DetNet Controller Plane without providing any explicit definition or
a reference of the latter: 

1.	It seems that the relevant definition appears in Section 4.4.2 of
RFC 8655

[Don]

Section 4.1., paragraph 1:

OLD:

    While the definition of controller plane for DetNet is out of the

    scope of this document, there are particular considerations and

    requirements for such that result from the unique characteristics of

    the DetNet architecture [I-D.ietf-detnet-architecture] and data plane

    as defined herein.

NEW:

    The Controller Plane corresponds to the aggregation of the Control

    and Management Planes discussed in [RFC7426] and [RFC8655].  While

    more details of any DetNet controller plane are out of the scope of

    this document, there are particular considerations and requirements

    for such that result from the unique characteristics of the DetNet

    architecture [RFC8655] and data plane as defined herein.

2.	IMHO adding this reference explicitly (or even reproducing it
verbatim) would help the reader to understand why, say, allocation and
distribution of S-labels and F-labels are required from the DetNet
Controller plane in the draft

[Don] added reference. see above.

7.	Section 3.6.1.4 “Network Coding” says that “Network Coding, not to
be confused with network programming, comprises several techniques where
multiple data flows are encoded”. 

1.	No further explanation or reference is provided
2.	I freely admit complete ignorance in all matters dealing with
Network Coding, therefore I did not understand why this section appears in
the draft. 
3.	One of the authors has pointed to the IRTF Coding for efficient
NetWork Communications Research Group (NWCRG)  as the possible source of
information about Network Coding, and this was really helpful to me. I think
that adding at least this pointer (and possibly some others) as Informative
Reference(s) to the draft,  would be helpful  to other readers as well. 

[Don] added reference.  

Section 3.6.1.4., paragraph 1:

OLD:

    Network Coding, not to be confused with network programming,

    comprises several techniques where multiple data flows are encoded.

    These resulting flows can then be sent on different paths.  The

    encoding operation can combine flows and error recovery information.

    When the encoded flows are decoded and recombined the original flows

    can be recovered.  Note that Network coding uses an alternative to

    packet by packet PREOF.  Therefore, for certain network topologies

    and traffic loads, Network Coding can be used to improve a network's

    throughput, efficiency, latency, and scalability, as well as

    resilience to partition, attacks, and eavesdropping, as compared to

    traditional methods.  DetNet could utilized Network coding as an

    alternative to other protection means.  Network coding is often

    applied in wireless networks and is being explored for other network

    types.

NEW:

    Network Coding, [nwcrg] not to be confused with network programming,

    comprises several techniques where multiple data flows are encoded.

    These resulting flows can then be sent on different paths.  The

    encoding operation can combine flows and error recovery information.

    When the encoded flows are decoded and recombined the original flows

    can be recovered.  Note that Network coding uses an alternative to

    packet by packet PREOF.  Therefore, for certain network topologies

    and traffic loads, Network Coding can be used to improve a network's

    throughput, efficiency, latency, and scalability, as well as

    resilience to partition, attacks, and eavesdropping, as compared to

    traditional methods.  DetNet could utilized Network coding as an

    alternative to other protection means.  Network coding is often

    applied in wireless networks and is being explored for other network

    types.

Section 7., paragraph 19:

OLD:

    [RFC2205]  Braden, R., Ed., Zhang, L., Berson, S., Herzog, S., and S.

               Jamin, "Resource ReSerVation Protocol (RSVP) -- Version 1

               Functional Specification", RFC 2205, DOI 10.17487/RFC2205,

               September 1997, <https://www.rfc-editor.org/info/rfc2205>.

NEW:

    [nwcrg]    IRTF, "Coding for efficient NetWork Communications

               Research Group (nwcrg)",

               <https://datatracker.ietf.org/rg/nwcrg/about>.

    [RFC2205]  Braden, R., Ed., Zhang, L., Berson, S., Herzog, S., and S.

               Jamin, "Resource ReSerVation Protocol (RSVP) -- Version 1

               Functional Specification", RFC 2205, DOI 10.17487/RFC2205,

               September 1997, <https://www.rfc-editor.org/info/rfc2205>.

Nits: 

I did not run the nits check on the draft. So I just included two sentences
that looked problematic to me. However, I am not a native English speaker,
so please consider these with a grain of salt.

1.	Something seems to be wrong with the grammar in the following
sentence in Section 3.6.1.1: “Misbehaving DetNet flows must be detected and
it have to be ensured that they do not compromise QoS of other flows”

[Don]

Section 3.6.1.1., paragraph 1:

OLD:

    Reservation of resources can allocate resources to specific DetNet

    flows.  This can eliminate packet contention and packet loss for

    DetNet traffic.  This also can reduce jitter for DetNet traffic.

    Resources allocated to a DetNet flow protect it from other traffic

    flows.  On the other hand, DetNet flows are assumed to behave with

    respect to the reserved traffic profile.  Misbehaving DetNet flows

    must be detected and it have to be ensured that they do not

    compromise QoS of other flows.  The use of (queuing, policing,

    shaping) policies can be used to ensure that the allocation of

    resources reserved for DetNet is met.

NEW:

    Reservation of resources can allocate resources to specific DetNet

    flows.  This can eliminate packet contention and packet loss for

    DetNet traffic.  This also can reduce jitter for DetNet traffic.

    Resources allocated to a DetNet flow protect it from other traffic

    flows.  On the other hand, DetNet flows are assumed to behave with

    respect to the reserved traffic profile.  Misbehaving DetNet flows

    must be able to be detected and ensure that they do not compromise

    QoS of other flows.  Queuing, policing, and shaping policies can be

    used to ensure that the allocation of resources reserved for DetNet

    is met.

2.	The next sentence in the same section “The use of (queuing,
policing, shaping) policies can be used to ensure that the allocation of
resources reserved for DetNet is met” IMHO should be rephrased e.g., like
“Queuing, policing, and shaping policies can be used to ensure that the
allocation of resources reserved for DetNet is met”. 

[Don] Don Please See above. 

 

Regards,

Sasha

Office: +972-39266302

Cell:      +972-549266302

Email:   [email-masked]
<mailto:[email-masked]> 

This e-mail message is intended for the recipient only and contains
information which is 
CONFIDENTIAL and which may be proprietary to ECI Telecom. If you have
received this 
transmission in error, please inform us by e-mail, phone or fax, and then
delete the original 
and all copies thereof.
___________________________________________________________________________

---

From: Alexander Vainshtein <[email-masked]>
Date: Tue, 14 Jan 2020 08:10:01 +0000

Don,
Lots of thanks for a very detailed response to my review.

I will read it carefully and provide my feedback, but it will take some time.

Regards, and apologies,
Sasha

Office: +972-39266302
Cell:      +972-549266302

-----Original Message-----
From: rtg-dir <[email-masked]> On Behalf Of Don Fedyk
Sent: Tuesday, January 14, 2020 12:40 AM
To: Alexander Vainshtein <[email-masked]>; 'Balázs Varga A' <[email-masked]>; [email-masked]
Cc: [email-masked]; [email-masked]; 'Yemin (Amy)' <[email-masked]>; [email-masked]
Subject: Re: [RTG-DIR] [Detnet] RTG-DIR last call review of draft-ietf-detnet-data-plane-framework-03

Hi Sasha 

comments one by one see[Don] with old/new.  If there are additional changes on your items it might

be good just to break off any item in a separate follow-up. I have included a

full side by side diff so you can examine the changes in place. (hopefully this will make through the mail filters).

If not I can send the complete proposed updated draft text – just let us know.  

Cheers

Don

From: Alexander Vainshtein <[email-masked] <mailto:[email-masked]> >
Sent: Thursday, December 26, 2019 5:30 PM
To: [email-masked] <mailto:[email-masked]>
Cc: [email-masked] <mailto:[email-masked]> ; [email-masked] <mailto:[email-masked]> ; [email-masked]
<mailto:[email-masked]> ; Yemin
(Amy) <[email-masked] <mailto:[email-masked]> >
Subject: RTG-DIR last call review of
draft-ietf-detnet-data-plane-framework-03

Hello, 

​https://clicktime.symantec.com/3VVqmKSjruaw9PhzaZVyfUd6H2?u=http%3A%2F%2Ftrac.tools.ietf.org%2Farea%2Frtg%2Ftrac%2Fwiki%2FRtgDir 

Summary: 

Comments:

Section 7., paragraph 3:

OLD:

NEW:

    document:

       Don Fedyk

       Jouni Korhonen

Minor Issues: 

OLD:

NEW:

               <https://clicktime.symantec.com/3VEZfYn6w8e4XTFbD3uUbAw6H2?u=https%3A%2F%2Fwww.rfc-editor.org%2Finfo%2Frfc8655>.

a.	On one hand:

                                                  i.     The DetNet Data

                                                 ii.     Section 3.2.2.2 of

Section 1., paragraph 5:

OLD:

    enhanced to support TSN.

NEW:

OLD:

NEW:

    the service sub-layer.

                                                  i.     Section 4.2 of RFC
4385

                                                 ii.     Section 7.3.2 of
RFC 4197

Section 3., paragraph 7:

OLD:

    Section 4.3.

NEW:

OLD:

NEW:

    account.

[Don] Clarified 

Section 1., paragraph 3:

OLD:

    delivery).

NEW:

OLD:

NEW:

    the service sub-layer.

                                                  i.     The relevant text

                                                 ii.     Personally I

[Don] 

OLD:

NEW:

[Don] Definitions 

    CW            Control Word.

OLD:

NEW:

OLD:

    [I-D.ietf-detnet-mpls].

NEW:

    [I-D.ietf-detnet-mpls].

                                                 ii.     I am not sure if

[Don] See above changes.

[Don]

OLD:

    as defined herein.

NEW:

[Don] added reference.  

OLD:

    types.

NEW:

    types.

OLD:

               September 1997, <https://clicktime.symantec.com/3XxTcXqWT3hkMHngS4hHBKn6H2?u=https%3A%2F%2Fwww.rfc-editor.org%2Finfo%2Frfc2205>.

NEW:

               Research Group (nwcrg)",

               <https://clicktime.symantec.com/37UcmCXykNagWaH6EXciECB6H2?u=https%3A%2F%2Fdatatracker.ietf.org%2Frg%2Fnwcrg%2Fabout>.

               September 1997, <https://clicktime.symantec.com/3XxTcXqWT3hkMHngS4hHBKn6H2?u=https%3A%2F%2Fwww.rfc-editor.org%2Finfo%2Frfc2205>.

Nits: 

[Don]

OLD:

NEW:

    is met.

 

 

Regards,

Sasha

Office: +972-39266302

Cell:      +972-549266302

---

From: Alexander Vainshtein <[email-masked]>
Date: Tue, 21 Jan 2020 11:42:11 +0000

Don,

First of all, apologies for a much delayed response.

Unfortunately I could not open the diff you've sent.

If you can re-send it in some commonly readable other format, it would be great (HTML or even PDF would definitely be the best).

Otherwise please send the new version of the draft as plain text.

Meanwhile please see some preliminary comments to your answers inline below.

It seems that most issues I’ve raised in my review have been successfully resolved.

Regards, and, again, apologies for the delay,

Sasha

Office: +972-39266302

Cell:      +972-549266302

-----Original Message-----
From: rtg-dir <[email-masked]> On Behalf Of Don Fedyk
Sent: Tuesday, January 14, 2020 12:40 AM
To: Alexander Vainshtein <[email-masked]>; 'Balázs Varga A' <[email-masked]>; [email-masked]
Cc: [email-masked]; [email-masked]; 'Yemin (Amy)' <[email-masked]>; [email-masked]
Subject: Re: [RTG-DIR] [Detnet] RTG-DIR last call review of draft-ietf-detnet-data-plane-framework-03

Hi Sasha

Cheers

Don

From: Alexander Vainshtein <[email-masked] <mailto:[email-masked]<mailto:[email-masked]%20%3cmailto:[email-masked]>> >

To: [email-masked]<mailto:[email-masked]> <mailto:[email-masked]>

Cc: [email-masked]<mailto:[email-masked]> <mailto:[email-masked]> ; [email-masked]<mailto:[email-masked]> <mailto:[email-masked]> ; [email-masked]<mailto:[email-masked]>

(Amy) <[email-masked] <mailto:[email-masked]<mailto:[email-masked]%20%3cmailto:[email-masked]>> >

Hello,

Review Date: 26-Dec-19

Summary:

Comments:

page of  the draft.

[Don] Two authors are now contributors. [[Sasha]] As I have said, this is between the team of the authors/contributors and the ADs, I am OK with the proposed resolution.

Section 7., paragraph 3:

OLD:

NEW:

    document:

       Don Fedyk

       Jouni Korhonen

Minor Issues:

to such a change.

[Don]Normative Reference. Done[[Sasha]] OK

OLD:

NEW:

a.            On one hand:

                                                  i.     The DetNet Data

                                                 ii.     Section 3.2.2.2 of

Framework draft).

[Don] Ordering is at the server [[Sasha]] Did you mean “Service” here? sub-layer. Resources - buffers for

Section 1., paragraph 5:

OLD:

    enhanced to support TSN.

NEW:

[[Sasha]]  The change above seems to address my other question (impact of non-support of TSN frame pre-emption inn other forwarding technologies.

[[Sasha]] I still wonder however, whether “typically not” is strong enough, or we can state that frame pre-emption is a unique capability of TSN?

OLD:

NEW:

   [[Sasha]]  The new text seems to begin here.

    through the network.  See [I-D.ietf-detnet-ip] [[Sasha]] Can you please provide a reference to the specific text in this draft? and Section 4.1 for

    the service sub-layer.

                                                  i.     Section 4.2 of RFC

4385

                                                 ii.     Section 7.3.2 of

RFC 4197

Section 3., paragraph 7:

OLD:

    Section 4.3.

NEW:

    Packets

[[Sasha]] Are sequence numbers always added to packets in DetNet?

Quoting from the detnet-ip draft<https://tools.ietf.org/html/draft-ietf-detnet-ip-04>, section 4.2:

   As noted earlier, service protection must be implemented within each
   link / sub-network independently, using the domain specific
   mechanisms.  This is due to the lack of unified end-to-end sequencing
   information that could be used by the intermediate nodes.

This suggests to me that DetNet over IP data plane does not add any sequence numbers to the packets.

And the detnet--mpls draft<https://tools.ietf.org/html/draft-ietf-detnet-mpls-04> in Section 4.2.1 includes the sequence number space of 0 (zero) bits as MUST to support (along with 16 bits and 28 bits space), effectively making it possible not to add a meaningful sequence number.

  enabling a range of packet order protection from simple

linkage in the draft.

OLD:

NEW:

    account.

[[Sasha]] Looks OK.

to see:

[Don] Clarified

Section 1., paragraph 3:

OLD:

    delivery).

NEW:

OLD:

NEW:

    the service sub-layer.

                                                  i.     The relevant text

as well as bridges”

                                                 ii.     Personally I

[Don] Hopefully above text changes cover this.[[Sasha]] OK

unidirectional.

useful

[Don]

OLD:

NEW:

[[Sasha]] After re-reading 4.2.4 I understand that the most relevant part of it is para 3 (beginning with the words “DetNet's use of PREOF”.

With this understanding in mind the new text looks stuffiest.

Data Plane: MPLS

[Don] Definitions

    CW            Control Word.

be added IMHO

OLD:

NEW:

OLD:

    [I-D.ietf-detnet-mpls].

NEW:

    [I-D.ietf-detnet-mpls].

                                                 ii.     I am not sure if

[Don] Left as informative.[[Sasha]] OK

[Don] See above changes.[[Sasha]] Please consider changing this to “a shim layer called the DetNet control word (dCW)”  followed by a reference to the detnet-mpls draft.

RFC 8655

[Don]

OLD:

    as defined herein.

NEW:

[Don] added reference. see above.[[Sasha]] OK

the draft.

[Don] added reference.

OLD:

    types.

NEW:

    types.

OLD:

NEW:

               Research Group (nwcrg)",

[[Sasha]] OK

Nits:

[Don]

OLD:

NEW:

    is met.

[[Sasha]] OK.

Regards,

Sasha

Office: +972-39266302

Cell:      +972-549266302

information which is

received this

delete the original

and all copies thereof.

---

From: "Don Fedyk" <[email-masked]>
Date: Wed, 29 Jan 2020 14:07:34 -0500

Hi Sasha

Sorry, now I had some travel.  I’m attaching the pdf of the diff (since
html I attached failed).  (The PDF seems to lose the color highlighting).  I
attached the modified file and you can do your own diff. 

Note that my comments are still of the Jan 13th date.  I will address your
points where noted below.  And then provide an updated draft and diff.

So you may want to wait for the update. 

Comments/clarifications below. [Don]

Also I have snipped out agreed sections for brevity.  

Thanks,

Don 

From: detnet <[email-masked]> On Behalf Of Alexander Vainshtein
Sent: Tuesday, January 21, 2020 6:42 AM
To: Don Fedyk <[email-masked]>
Cc: [email-masked]; 'Yemin (Amy)' <[email-masked]>;
[email-masked]; 'Balázs Varga A' <[email-masked]>;
[email-masked]; [email-masked]
Subject: Re: [Detnet] [RTG-DIR] RTG-DIR last call review of
draft-ietf-detnet-data-plane-framework-03

Don,

[Don] No worries

If you can re-send it in some commonly readable other format, it would be
great (HTML or even PDF would definitely be the best).

Meanwhile please see some preliminary comments to your answers inline
below.

It seems that most issues I’ve raised in my review have been successfully
resolved.

Sasha

Office: +972-39266302

Cell:      +972-549266302

<snip>

[Don] Two authors are now contributors. [[Sasha]] As I have said, this is
between the team of the authors/contributors and the ADs, I am OK with the
proposed resolution.

[Don] Agreed

<snip>

2.            After reading both  RFC 8655 and the DetNet Data Plane
Framework

(POF)” is expected to actually reorder (or try to reorder) packets that
have

a.            On one hand:

                                                  i.     The DetNet Data

                                                 ii.     Section 3.2.2.2 of

b.            On the other hand, neither of these documents mentions the
need for

of packets received out of order, and impact of this operation on the
packet

associated just with the forwarding sub-layer (see Figure 2 in Section
4.1.1

Framework draft).

c.             For comparison, the PWE3 documents that deal with sequencing
and

[Don] Ordering is at the server [[Sasha]] Did you mean “Service” here?
sub-layer. Resources - buffers for

[Don] Yes Service Updated text delineates where buffer resources are used.
Section 1., paragraph 5:

OLD:

    enhanced to support TSN.

NEW:

[[Sasha]]  The change above seems to address my other question (impact of
non-support of TSN frame pre-emption inn other forwarding technologies.

[[Sasha]] I still wonder however, whether “typically not” is strong enough,
or we can state that frame pre-emption is a unique capability of TSN?

[Don] Well I am aware of other technologies that used preemption and
internet search say  RFC2686 and RFC2687 is one example.  The wording was
intentional on my part. 

Also we do not want to preclude other data planes that may have the
capability.   

OLD:

 

NEW:

    through the network.  See [I-D.ietf-detnet-ip] [[Sasha]] Can you please
provide a reference to the specific text in this draft? and Section 4.1 for

[Don] Will add a section reference.

    the service sub-layer.

                                                  i.     Section 4.2 of RFC

4385

<https://clicktime.symantec.com/33EPoJPGK3P16uEhFVxEQPF6H2?u=https%3A%2F%2F
t

dropping or reordering an out-of-sequence packet is at the discretion of
the

                                                 ii.     Section 7.3.2 of

RFC 4197

<https://clicktime.symantec.com/37VT1KAX7zdhNmeuVcyLo5c6H2?u=https%3A%2F%2F
t

d.            From my POV the DetNet Data Plane Framework draft should
clearly

define the expectations from the POF regarding handling of packets that
have

Section 3., paragraph 7:

OLD:

    Section 4.3.

NEW:

    Packets

[Don] Nope

Quoting from the  <https://tools.ietf.org/html/draft-ietf-detnet-ip-04>
detnet-ip draft, section 4.2:

This suggests to me that DetNet over IP data plane does not add any
sequence numbers to the packets.

[Don] DetNet over IP does not add a sequence number but the DetNet
architecture states that sequence numbers at higher Layers may be utilized
for example IPsec.    

And the  <https://tools.ietf.org/html/draft-ietf-detnet-mpls-04>
detnet--mpls draft in Section 4.2.1 includes the sequence number space of 0
(zero) bits as MUST to support (along with 16 bits and 28 bits space),
effectively making it possible not to add a meaningful sequence number.

[Don] Will look to see if there is something I can clarify but if you have
specific text please point it out. 

<snip>

5.            The draft states, in Section 4.2.4., that “DetNet
applications

<https://clicktime.symantec.com/3S5RkymWb9AHCmyh98pDnNf6H2?u=https%3A%2F%2F
t

<https://clicktime.symantec.com/32QD67HM2j2fhREf17yZBwp6H2?u=https%3A%2F%2F
t

unidirectional. 

2.            I think that some clarification, preferably with references
to

useful

[Don] 

OLD:

 

NEW:

[[Sasha]] After re-reading 4.2.4 I understand that the most relevant part
of it is para 3 (beginning with the words “DetNet's use of PREOF”.
With this understanding in mind the new text looks stuffiest. 
[Don] stuffiest = to suffice?  Stuffiest makes me chuckle though. 😊

<snip>

---

From: Alexander Vainshtein <[email-masked]>
Date: Thu, 30 Jan 2020 07:07:40 +0000

Don,
Lots of thanks for taking care of that.
I will look carefully at the differences and provide a detailed response a in a few days.

Regards,
Sasha

Office: +972-39266302
Cell:      +972-549266302

From: Don Fedyk <[email-masked]>
Sent: Wednesday, January 29, 2020 9:08 PM
To: Alexander Vainshtein <[email-masked]>
Cc: [email-masked]; 'Yemin (Amy)' <[email-masked]>; [email-masked]; 'Balázs Varga A' <[email-masked]>; [email-masked]; [email-masked]
Subject: RE: [Detnet] [RTG-DIR] RTG-DIR last call review of draft-ietf-detnet-data-plane-framework-03

Hi Sasha

Sorry, now I had some travel.  I’m attaching the pdf of the diff (since html I attached failed).  (The PDF seems to lose the color highlighting).  I attached the modified file and you can do your own diff.
Note that my comments are still of the Jan 13th date.  I will address your points where noted below.  And then provide an updated draft and diff.
So you may want to wait for the update.

Thanks,
Don

From: detnet <[email-masked]<mailto:[email-masked]>> On Behalf Of Alexander Vainshtein
Sent: Tuesday, January 21, 2020 6:42 AM
To: Don Fedyk <[email-masked]<mailto:[email-masked]>>
Cc: [email-masked]<mailto:[email-masked]>; 'Yemin (Amy)' <[email-masked]<mailto:[email-masked]>>; [email-masked]<mailto:[email-masked]>; 'Balázs Varga A' <[email-masked]<mailto:[email-masked]>>; [email-masked]<mailto:[email-masked]>; [email-masked]<mailto:[email-masked]>
Subject: Re: [Detnet] [RTG-DIR] RTG-DIR last call review of draft-ietf-detnet-data-plane-framework-03

Don,

[Don] No worries

Sasha

Office: +972-39266302

Cell:      +972-549266302

<snip>

[Don] Agreed

<snip>

a.            On one hand:

                                                  i.     The DetNet Data

                                                 ii.     Section 3.2.2.2 of

Framework draft).

Section 1., paragraph 5:

OLD:

    enhanced to support TSN.

NEW:

[Don] Well I am aware of other technologies that used preemption and internet search say  RFC2686 and RFC2687 is one example.  The wording was intentional on my part.

Also we do not want to preclude other data planes that may have the capability.

OLD:

NEW:

    the service sub-layer.

                                                  i.     Section 4.2 of RFC

4385

                                                 ii.     Section 7.3.2 of

RFC 4197

Section 3., paragraph 7:

OLD:

    Section 4.3.

NEW:

    Packets

[Don] Nope

[Don] DetNet over IP does not add a sequence number but the DetNet architecture states that sequence numbers at higher Layers may be utilized for example IPsec.

[Don] Will look to see if there is something I can clarify but if you have specific text please point it out.

<snip>

unidirectional.

useful

[Don]

OLD:

NEW:

<snip>