Subject: [Ioam] IOAM in IPPM

From: "Frank Brockners (fbrockne)" <[email-masked]>
Date: Thu, 16 Feb 2017 17:06:39 +0000

Dear IPPM WG,

following the recommendation from the IPPM chairs (see also Brian's email below) as well as ADs involved (see also Alvaro's email below) the discussions on in-situ OAM (IOAM), we'd like the IPPM WG to consider adopting the IOAM work, especially the definition of formats and associated procedures for in-situ OAM, including mechanisms for capturing path and path-traversal related information as well as procedures to employ, configure, trigger, and export the embedded telemetry information. For this we'd suggest to adopt https://tools.ietf.org/html/draft-brockners-inband-oam-data-02 as a starting point for the work of IPPM on IOAM. Like Brian noted, IPPM WG already focusses on hybrid measurements (e.g. those enabled by the similar draft-ietf-ippm-6man-pdm-option), hence IOAM would be a natural complement and addition to that work and would benefit from all earlier work on metrics etc. 

Background on In-situ OAM: In-situ OAM provides real-time telemetry of individual data packets and flows. It is based on telemetry information which is embedded within live user traffic, where "live user traffic" means packets originated and terminated at the application layer. For more information on in-situ OAM, you could refer to the requirements draft we posted (draft-brockners-inband-oam-requirements-02), or check out the presentations we gave at IETF 96 and 97: https://www.ietf.org/proceedings/96/slides/slides-96-opsawg-8.pdf and https://www.ietf.org/proceedings/97/slides/slides-97-opsawg-in-situ-oam-00.PDF 

Appreciate your thoughts and support :-).

Thanks, Frank

-----Original Message-----
From: Brian Trammell (IETF) [mailto:[email-masked]] 
Sent: Donnerstag, 16. Februar 2017 16:13
To: Frank Brockners (fbrockne) <[email-masked]>
Subject: IOAM in IPPM

Hi, Frank,

As we discussed today, please do introduce your draft (draft-brockners-inband-oam-data-00) to the IPPM mailing list. Given our recent focus on hybrid measurements (e.g. those enabled by the similar draft-ietf-ippm-6man-pdm-option), I'd like to start a discussion there about considering the draft for adoption within IPPM.

Thanks, cheers,

Brian

-----Original Message-----
From: Ioam [mailto:[email-masked]] On Behalf Of Alvaro Retana (aretana)
Sent: Donnerstag, 16. Februar 2017 17:58
To: [email-masked]
Subject: [Ioam] IOAM Work Moving Forward

Hi!

First of all, thank you all for the interest expressed in this topic and the discussions about the charter.

As you know, one of the discussions resulting from the Internal Review of the proposed IOAM charter was whether the work was already within the ippm WG scope or not.  Over the last couple of days, I have dug deeper into that question with the Transport ADs and the ippm WG Chairs, and our conclusion is that there is significant overlap between the current ippm Charter and the proposed IOAM work.  Enough to justify moving the work forward in the ippm WG and not splintering a related effort into a new WG.

I have then stopped the chartering effort for a new WG.  The proponents will start discussions on the ippm list soon - please join if you're not there already.  I will keep this list open for a couple more days.

Clearly there is interest in developing this cross-area work in the IETF.  I hope that you will continue to participate as the topic progresses on the ippm WG.  It is important that this type of cross-area efforts be properly discussed and that we don't create more silos.  I realize it took us a couple of tries to find what I think is a stable home for the IOAM work - I know that the open discussion has only helped the overall process.

Thanks!

Alvaro.

---

From: <[email-masked]>
Date: Thu, 16 Feb 2017 19:15:11 +0000 (UTC)

Frank,

As one of the authors of the PDM draft (draft-ietf-ippm-6man-pdm-option) that you discussed below.  I find your work extremely interesting, as you know!

I hope to see your work in IPPM.  I can see that we may be able to collaborate and do quite interesting work in the future.
 Thanks,

Nalini Elkins
Inside Products, Inc.
www.insidethestack.com
(831) 659-8360

----- Original Message -----
From: Frank Brockners (fbrockne) <[email-masked]>
To: "[email-masked]" <[email-masked]>
Cc: "[email-masked]" <[email-masked]>
Sent: Thursday, February 16, 2017 9:06 AM
Subject: [ippm] IOAM in IPPM

Dear IPPM WG,

Thanks, Frank

-----Original Message-----
From: Brian Trammell (IETF) [mailto:[email-masked]] 
Sent: Donnerstag, 16. Februar 2017 16:13
To: Frank Brockners (fbrockne) <[email-masked]>
Subject: IOAM in IPPM

Hi, Frank,

Thanks, cheers,

Brian

-----Original Message-----
From: Ioam [mailto:[email-masked]] On Behalf Of Alvaro Retana (aretana)
Sent: Donnerstag, 16. Februar 2017 17:58
To: [email-masked]
Subject: [Ioam] IOAM Work Moving Forward

Hi!

Thanks!

Alvaro.
_______________________________________________
ippm mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/ippm

---

From: "MORTON, ALFRED C (AL)" <[email-masked]>
Date: Thu, 16 Feb 2017 18:57:25 +0000

Hi Frank and numerous IOAM co-authors, 

(with due respect to the charter discussion thread)

When I saw the IOAM mailing list announced a few weeks back
I looked-in and saw that there were several terms used to categorize
this type of measurement method (in-situ, in-band, etc.).
I planned to send a pointer to the definitions of categories 
IPPM (and IETF) agreed when new methods began to appear,
so I'll do it now: https://tools.ietf.org/html/rfc7799

>From the slides and the early sections of the draft,
it appears what you propose here is a Hybrid Type I method
(Augmentation or modification of the stream of interest).
This proposal has much in common with the PDM Destination
Option Header: https://tools.ietf.org/html/draft-ietf-ippm-6man-pdm-option-07
which we knew about and classified in the RFC:
https://tools.ietf.org/html/rfc7799#section-4.2

   This method processes a user traffic stream and adds "fields which
   are dedicated to measurement" (the measurement intent is made clear
   in the title of this option).

It may be useful to look at the points about MTU made in 
section 4.2. It seems that the user traffic entering the 
in-situ domain will need to leave significant space for 
the various data format fields that would be added. If this
topic has already been discussed, I hope you'll summarize it
here for the IPPM crowd and/or provide good pointers.

Another goal of Hybrid measurement is that modified traffic
and unmodified traffic would be treated as the *same* class
of traffic by the network. Otherwise, one advantage over 
active measurement methods would be lost.  We described this
problem at the end of page 7 in RFC 7799: 

      Hybrid Methods of measurement that augment or modify packets of a
      "class C" in a host should produce results equivalent to Passive
      Methods of Measurement when hosts accessing and links transporting
      these packets along the path (other than those performing
      augmentation/modification) treat packets from both categories of
      methods (with and without the augmentation/modification) as the
      same "class C".  The Passive Methods of Measurement represent the
      Ground Truth when comparing results between Passive and Hybrid
      Methods, and this comparison should be conducted to confirm the
      "class C" treatment.

I wonder if you have considered this comparison/constraint yet during the
IOAM format and measurement development. It would likely be useful
to cover this topic on IPPM list.

thanks and regards,
Al

> From: ippm [mailto:[email-masked]] On Behalf Of Frank Brockners
> (fbrockne)
> Sent: Thursday, February 16, 2017 12:07 PM
> To: [email-masked]
> Cc: [email-masked]
> following the recommendation from the IPPM chairs (see also Brian's
> email below) as well as ADs involved (see also Alvaro's email below) the
> discussions on in-situ OAM (IOAM), we'd like the IPPM WG to consider
> adopting the IOAM work, especially the definition of formats and
> associated procedures for in-situ OAM, including mechanisms for
> capturing path and path-traversal related information as well as
> procedures to employ, configure, trigger, and export the embedded
> telemetry information. For this we'd suggest to adopt
> https://tools.ietf.org/html/draft-brockners-inband-oam-data-02 as a
> starting point for the work of IPPM on IOAM. Like Brian noted, IPPM WG
> already focusses on hybrid measurements (e.g. those enabled by the
> similar draft-ietf-ippm-6man-pdm-option), hence IOAM would be a natural
> complement and addition to that work and would benefit from all earlier
> work on metrics etc.
> 
> Background on In-situ OAM: In-situ OAM provides real-time telemetry of
> individual data packets and flows. It is based on telemetry information
> which is embedded within live user traffic, where "live user traffic"
> means packets originated and terminated at the application layer. For
> more information on in-situ OAM, you could refer to the requirements
> draft we posted (draft-brockners-inband-oam-requirements-02), or check
> out the presentations we gave at IETF 96 and 97:
> https://www.ietf.org/proceedings/96/slides/slides-96-opsawg-8.pdf and
> https://www.ietf.org/proceedings/97/slides/slides-97-opsawg-in-situ-oam-
> 00.PDF
> 
> As we discussed today, please do introduce your draft (draft-brockners-
> inband-oam-data-00) to the IPPM mailing list. Given our recent focus on
> hybrid measurements (e.g. those enabled by the similar draft-ietf-ippm-
> 6man-pdm-option), I'd like to start a discussion there about considering
> the draft for adoption within IPPM.
> 
> Thanks, cheers,
> 
> Brian
> 
> As you know, one of the discussions resulting from the Internal Review
> of the proposed IOAM charter was whether the work was already within the
> ippm WG scope or not.  Over the last couple of days, I have dug deeper
> into that question with the Transport ADs and the ippm WG Chairs, and
> our conclusion is that there is significant overlap between the current
> ippm Charter and the proposed IOAM work.  Enough to justify moving the
> work forward in the ippm WG and not splintering a related effort into a
> new WG.
> 
> I have then stopped the chartering effort for a new WG.  The proponents
> will start discussions on the ippm list soon - please join if you're not
> there already.  I will keep this list open for a couple more days.
> 
> Clearly there is interest in developing this cross-area work in the
> IETF.  I hope that you will continue to participate as the topic
> progresses on the ippm WG.  It is important that this type of cross-area
> efforts be properly discussed and that we don't create more silos.  I
> realize it took us a couple of tries to find what I think is a stable
> home for the IOAM work - I know that the open discussion has only helped
> the overall process.
> 
> Thanks!
> 
> Alvaro.

---

From: "Frank Brockners (fbrockne)" <[email-masked]>
Date: Fri, 17 Feb 2017 11:14:08 +0000

Hi Al,

thanks for your notes - please see inline (..FB:)

-----Original Message-----
From: MORTON, ALFRED C (AL) [mailto:[email-masked]] 
Sent: Donnerstag, 16. Februar 2017 19:57
To: Frank Brockners (fbrockne) <[email-masked]>; [email-masked]
Cc: [email-masked]
Subject: RE: IOAM in IPPM

When I saw the IOAM mailing list announced a few weeks back I looked-in and saw that there were several terms used to categorize this type of measurement method (in-situ, in-band, etc.).
I planned to send a pointer to the definitions of categories IPPM (and IETF) agreed when new methods began to appear, so I'll do it now: https://tools.ietf.org/html/rfc7799

>From the slides and the early sections of the draft, it appears what you propose here is a Hybrid Type I method (Augmentation or modification of the stream of interest).
This proposal has much in common with the PDM Destination Option Header: https://tools.ietf.org/html/draft-ietf-ippm-6man-pdm-option-07
which we knew about and classified in the RFC:
https://tools.ietf.org/html/rfc7799#section-4.2

...FB: Yes. In-situ OAM would be "Hybrid OAM, Type 1" - we also mention this explicitly in the requirements draft and refer to RFC7799, see section 1 in https://tools.ietf.org/html/draft-brockners-inband-oam-requirements-02#page-3

It may be useful to look at the points about MTU made in section 4.2. It seems that the user traffic entering the in-situ domain will need to leave significant space for the various data format fields that would be added. If this topic has already been discussed, I hope you'll summarize it here for the IPPM crowd and/or provide good pointers.

...FB: In-situ OAM information requires space in the packet - and the in-situ OAM needs to have appropriate MTU support. A discussion on MTU and packet-size is found in the requirements draft in section 4.2, see https://tools.ietf.org/html/draft-brockners-inband-oam-requirements-02#page-12

Another goal of Hybrid measurement is that modified traffic and unmodified traffic would be treated as the *same* class of traffic by the network. Otherwise, one advantage over active measurement methods would be lost.  We described this problem at the end of page 7 in RFC 7799: 

I wonder if you have considered this comparison/constraint yet during the IOAM format and measurement development. It would likely be useful to cover this topic on IPPM list.

...FB: This is a very worthwhile comment and has been briefly raised before during discussions in OPSAWG at the last IETF. There is no easy answer to the question: "Will a packet which is added in-situ OAM data be treated the same as if it would not be added in-situ OAM data?", because it'll likely depend on implementation details as well as the encapsulation used. Example: There are implementations which forward packets differently in case an IPv6 extension header is present in the packet - while not the case for all implementations, some have that behavior. Now if IOAM would be added as an extension header and it would be the only extension header present in the packet, some implementations might indeed change their forwarding behavior. Hence it is important to make sure that for the IOAM domain you indeed have nodes which conform to the assumption that forwarding behavior will not change when IOAM data is added to the packet.
What this leads to is probably another requirement we need to take into consideration - and that we should add to the requirements draft: E.g. something like "The addition of IOAM data-records should not change the way packets are forwarded *within* the IOAM domain."

Thoughts?

Thanks, Frank

thanks and regards,
Al

> email below) as well as ADs involved (see also Alvaro's email below) 
> the discussions on in-situ OAM (IOAM), we'd like the IPPM WG to 
> consider adopting the IOAM work, especially the definition of formats 
> and associated procedures for in-situ OAM, including mechanisms for 
> similar draft-ietf-ippm-6man-pdm-option), hence IOAM would be a 
> natural complement and addition to that work and would benefit from 
> all earlier work on metrics etc.
> 
> individual data packets and flows. It is based on telemetry 
> information which is embedded within live user traffic, where "live user traffic"
> inband-oam-data-00) to the IPPM mailing list. Given our recent focus 
> on hybrid measurements (e.g. those enabled by the similar 
> draft-ietf-ippm- 6man-pdm-option), I'd like to start a discussion 
> there about considering the draft for adoption within IPPM.
> 
> Thanks, cheers,
> 
> Brian
> 
> of the proposed IOAM charter was whether the work was already within 
> the ippm WG scope or not.  Over the last couple of days, I have dug 
> deeper into that question with the Transport ADs and the ippm WG 
> Chairs, and our conclusion is that there is significant overlap 
> between the current ippm Charter and the proposed IOAM work.  Enough 
> to justify moving the work forward in the ippm WG and not splintering 
> a related effort into a new WG.
> 
> I have then stopped the chartering effort for a new WG.  The 
> proponents will start discussions on the ippm list soon - please join 
> if you're not there already.  I will keep this list open for a couple more days.
> 
> progresses on the ippm WG.  It is important that this type of 
> cross-area efforts be properly discussed and that we don't create more 
> silos.  I realize it took us a couple of tries to find what I think is 
> a stable home for the IOAM work - I know that the open discussion has 
> only helped the overall process.
> 
> Thanks!
> 
> Alvaro.

---

From: "Frank Brockners (fbrockne)" <[email-masked]>
Date: Fri, 17 Feb 2017 11:45:13 +0000

Thanks Nalini. Looking forward to the IOAM work in IPPM. Frank

-----Original Message-----
From: Ioam [mailto:[email-masked]] On Behalf Of [email-masked]
Sent: Donnerstag, 16. Februar 2017 20:15
To: Frank Brockners (fbrockne) <[email-masked]>; [email-masked]
Cc: Robert Hamilton <[email-masked]>; [email-masked]; Michael Ackermann <[email-masked]>
Subject: Re: [Ioam] [ippm] IOAM in IPPM

Frank,

Nalini Elkins
Inside Products, Inc.
www.insidethestack.com
(831) 659-8360

----- Original Message -----
From: Frank Brockners (fbrockne) <[email-masked]>
To: "[email-masked]" <[email-masked]>
Cc: "[email-masked]" <[email-masked]>
Sent: Thursday, February 16, 2017 9:06 AM
Subject: [ippm] IOAM in IPPM

Dear IPPM WG,

Thanks, Frank

-----Original Message-----
From: Brian Trammell (IETF) [mailto:[email-masked]] 
Sent: Donnerstag, 16. Februar 2017 16:13
To: Frank Brockners (fbrockne) <[email-masked]>
Subject: IOAM in IPPM

Hi, Frank,

Thanks, cheers,

Brian

-----Original Message-----
From: Ioam [mailto:[email-masked]] On Behalf Of Alvaro Retana (aretana)
Sent: Donnerstag, 16. Februar 2017 17:58
To: [email-masked]
Subject: [Ioam] IOAM Work Moving Forward

Hi!

Thanks!

Alvaro.

---

From: "MORTON, ALFRED C (AL)" <[email-masked]>
Date: Sat, 18 Feb 2017 02:34:39 +0000

Hi Frank, one last reply, in-line [ACM]

> From: Frank Brockners (fbrockne) [mailto:[email-masked]]
> Sent: Friday, February 17, 2017 6:14 AM
> To: MORTON, ALFRED C (AL); [email-masked]
> Cc: [email-masked]
> When I saw the IOAM mailing list announced a few weeks back I looked-in
> and saw that there were several terms used to categorize this type of
> measurement method (in-situ, in-band, etc.).
> I planned to send a pointer to the definitions of categories IPPM (and
> IETF) agreed when new methods began to appear, so I'll do it now:
> https://tools.ietf.org/html/rfc7799
> 
> From the slides and the early sections of the draft, it appears what you
> propose here is a Hybrid Type I method (Augmentation or modification of
> the stream of interest).
> This proposal has much in common with the PDM Destination Option Header:
> https://tools.ietf.org/html/draft-ietf-ippm-6man-pdm-option-07
> ...FB: Yes. In-situ OAM would be "Hybrid OAM, Type 1" - we also mention
> this explicitly in the requirements draft and refer to RFC7799, see
> section 1 in https://tools.ietf.org/html/draft-brockners-inband-oam-
> requirements-02#page-3
[ACM] 
Thanks for the reference to IOAM requirements, I didn't see that one.
Glad we're in agreement.

> 
> It may be useful to look at the points about MTU made in section 4.2. It
> seems that the user traffic entering the in-situ domain will need to
> leave significant space for the various data format fields that would be
> added. If this topic has already been discussed, I hope you'll summarize
> it here for the IPPM crowd and/or provide good pointers.
> 
> ...FB: In-situ OAM information requires space in the packet - and the
> in-situ OAM needs to have appropriate MTU support. A discussion on MTU
> and packet-size is found in the requirements draft in section 4.2, see
> https://tools.ietf.org/html/draft-brockners-inband-oam-requirements-
> 02#page-12
> 
> Another goal of Hybrid measurement is that modified traffic and
> unmodified traffic would be treated as the *same* class of traffic by
> the network. Otherwise, one advantage over active measurement methods
> would be lost.  We described this problem at the end of page 7 in RFC
> 7799:
> 
> I wonder if you have considered this comparison/constraint yet during
> the IOAM format and measurement development. It would likely be useful
> ...FB: This is a very worthwhile comment and has been briefly raised
> before during discussions in OPSAWG at the last IETF. There is no easy
> answer to the question: "Will a packet which is added in-situ OAM data
> be treated the same as if it would not be added in-situ OAM data?",
> because it'll likely depend on implementation details as well as the
> encapsulation used. Example: There are implementations which forward
> packets differently in case an IPv6 extension header is present in the
> packet - while not the case for all implementations, some have that
> behavior. Now if IOAM would be added as an extension header and it would
> be the only extension header present in the packet, some implementations
> might indeed change their forwarding behavior. Hence it is important to
> make sure that for the IOAM domain you indeed have nodes which conform
> to the assumption that forwarding behavior will not change when IOAM
> data is added to the packet.
> What this leads to is probably another requirement we need to take into
> consideration - and that we should add to the requirements draft: E.g.
> something like "The addition of IOAM data-records should not change the
> way packets are forwarded *within* the IOAM domain."
> 
> Thoughts?
[ACM] 
Yes, the additional requirement on the domain is needed
We should also be able to specify one or more ways to 
check that the requirement is met on a representative sample
of paths across the domain.

I suggest tests with two packet streams, one unmodified and
another with incremental or pre-allocated insertion of the 
IOAM data records. The set of metrics planned for measurement
provide the basis for comparison, for example is the latency of
the unmodified stream always less than an IOAM modified stream?
We have to find a way to enable passive measurements on the 
unmodified stream, but that's a detail to work-out later.

Some of the other features of IOAM will be more challenging to
test in this way, but let's give it some more thought...

regards,
Al