Subject: [ippm] Review on draft-ietf-ippm-ioam-data-06

From: Jai Kumar <[email-masked]>
Date: Tue, 20 Aug 2019 19:22:16 -0700

Hello Frank,

This is in context of our conversation at IETF105. My goal is to provide input and improve current IOAM data draft with the learnings we had with IFA deployment.

This feedback is based on various customer interactions and concerns raised by them wrt IOAM. Each feedback is a longer topic and I am starting this thread as a summary email. This is just highlighting the issues and not yet proposing any solution. 

Feedback 1:

Section 4.2.1 Pre-allocated and Incremental Trace Options

Pre-allocated and incremental trace option is 8Bytes long. This can be easily reduced to 4Bytes.

There is a feedback that pre-allocated option is really not needed and either be removed or made optional.

Given that deployments are sensitive to the IOAM overhead (specially in 5G deployments), it’s a 50% fixed overhead savings on a per packet basis.

Feedback 2:
Section 4.1 IOAM Namespaces
Namespaces should be treated as templates (similar to IPFIX template record formats). This is more flexible way of enumerating data. 64K namespace id is a very large namespace and can be reduced to 64 IANA specified name spaces. Separate private name space can be allowed instead of interleaving of opaque data in the IANA allocated name space as suggested in the current draft “opaque state snapshot”.

https://tools.ietf.org/html/rfc7011#section-3.4

Feedback 3:

Section 4.2.1 Pre-allocated and Incremental Trace Options

IOAM-Trace-Type:  A 24-bit identifier which specifies which data

      types are used in this node data list.

This is the most contentious of all. In the current proposal, as new data fields are added, there is a corresponding trace type bit need in the header. This essentially means that all possible data fields need to be enumerated. Given that we there are 64K names spaces allowed, I don’t see how we can fit all possible data fields in this 24bit vector. I know there was a suggestion of keeping last bit as an extension bit but it is still scalable and/or easy to implement in hardware. Besides this the data fields are not annotated/encoded with the data type, something like in IPFIX https://tools.ietf.org/html/rfc7011#section-6.1

Feedback 4:

There is no version field in the data header and this will make interoperability challenging. Standard will evolve and headers bit definition and/or trace type will change and without version field HW will not be able to correctly handle the IOAM data headers.

Feedback 5:

Handling of TCP/UDP traffic using GRE encap is not acceptable. Here are some of the issues I can think of
GRE encaped IOAM packets will traverse a different network path then the original packet
Not all packets can be GRE encaped to avoid the previous problem, due to wastage of network bandwidth (typically sampled traffic is used for IOAM). What about native GRE traffic, will it get further encaped in another GRE tunnel and so forth.
IP header protocol will point to GRE IP proto and IOAM ethertype (pending allocation by IEEE) need to be read from the GRE header to detect an IOAM packet. This means parsing performance penalty for all regular GRE (non IOAM) traffic.

Thanks,

-Jai

---

From: "Carlos Pignataro (cpignata)" <[email-masked]>
Date: Wed, 21 Aug 2019 11:34:26 +0000

Hello, Vijay,

Please see https://datatracker.ietf.org/doc/draft-weis-ippm-ioam-eth/, and the document this replaces.

Thanks!

Thumb typed by Carlos Pignataro.
Excuze typofraphicak errows

2019/08/21 6:35、Vijay Rangarajan <[email-masked]<mailto:[email-masked]>>のメール:

Hello all:
Apologise if this has been previously discussed.
In reading "draft-ietf-ippm-ioam-data-06", I don't see mention of GRE encap. The draft, in fact in Sec 3, says the following - "The in-situ OAM data field can be transported by a variety of transport protocols, including NSH, Segment Routing, Geneve, IPv6, or IPv4.  Specification details for these different transport protocols are outside the scope of this document."

Is there another document, or a description somewhere, that talks about how IOAM is proposed to be carried in GRE? what would be the GRE payload, the GRE protocol type etc?

Thanks,
Vijay

On Wed, Aug 21, 2019 at 7:52 AM Jai Kumar <[email-masked]<mailto:[email-masked]>> wrote:
Hello Frank,

Feedback 1:

Feedback 2:

Feedback 3:

Feedback 4:

Feedback 5:

  *   GRE encaped IOAM packets will traverse a different network path then the original packet
  *   Not all packets can be GRE encaped to avoid the previous problem, due to wastage of network bandwidth (typically sampled traffic is used for IOAM). What about native GRE traffic, will it get further encaped in another GRE tunnel and so forth.
  *   IP header protocol will point to GRE IP proto and IOAM ethertype (pending allocation by IEEE) need to be read from the GRE header to detect an IOAM packet. This means parsing performance penalty for all regular GRE (non IOAM) traffic.

Thanks,
-Jai

---

From: Vijay Rangarajan <[email-masked]>
Date: Thu, 22 Aug 2019 18:01:00 +0530

Thanks Carlos, for pointing me to the draft.

Based on my understanding of the two drafts I have the following questions
and concerns:

   1. If I understand correctly, to deploy inband telemetry, we would need
   to construct GRE tunnels coinciding with the IOAM domain?
   2. GRE typically requires configuration to provision the tunnels.
   Provisioning and managing these tunnels and keeping these updated as the
   network grows/shrinks could be a significant overhead.
   3. In order to get the benefit of telemetry, we are imposing a change in
   forwarding protocol/topology and configuration - which, I feel is not
   desirable. For example, a customer might have basic L3 routing enabled and
   the expectation would be for inband telemetry to work seamlessly, without
   having to revamp the network with GRE tunnels and such. This could be a
   significant barrier to deployment.
   4. If sampling is used to select packets for performing IOAM encap, is
   the expectation that only sampled IOAM packets go through GRE encap? Or all
   data packets?
   5. Due to network nodes inserting the IOAM data, the inner L3/L4 headers
   keep getting pushed deeper. I would imagine this gets challenging for ASICs
   to access these fields for hashing/load balancing.
   6. Assuming only a subset of packets in a flow are subject to IOAM
   (based on sampling), how do we ensure these packets take the same network
   path as the rest of the packets in the flow?

Thanks,
Vijay

On Wed, Aug 21, 2019 at 5:04 PM Carlos Pignataro (cpignata) <
[email-masked]> wrote:

> 2019/08/21 6:35、Vijay Rangarajan <[email-masked]>のメール:
>
> Hello all:
> In reading "draft-ietf-ippm-ioam-data-06", I don't see mention of GRE
> encap. The draft, in fact in Sec 3, says the following - "The in-situ OAM
> data field can be transported by a variety of transport protocols,
> including NSH, Segment Routing, Geneve, IPv6, or IPv4.  Specification
> details for these different transport protocols are outside the scope of
> this document."
>
> Is there another document, or a description somewhere, that talks about
> how IOAM is proposed to be carried in GRE? what would be the GRE payload,
> the GRE protocol type etc?
>
> Thanks,
> Vijay
>
>
> On Wed, Aug 21, 2019 at 7:52 AM Jai Kumar <[email-masked]> wrote:
>
>> Hello Frank,
>>
>>
>>
>> This is in context of our conversation at IETF105. My goal is to provide
>> input and improve current IOAM data draft with the learnings we had with
>> IFA deployment.
>>
>> This feedback is based on various customer interactions and concerns
>> raised by them wrt IOAM. Each feedback is a longer topic and I am starting
>> this thread as a summary email. This is just highlighting the issues and
>> not yet proposing any solution.
>>
>>
>>
>>
>>
>> Feedback 1:
>>
>> Given that deployments are sensitive to the IOAM overhead (specially in
>> 5G deployments), it’s a 50% fixed overhead savings on a per packet basis.
>>
>>
>>
>>
>>
>> Feedback 2:
>> Namespaces should be treated as templates (similar to IPFIX template
>> record formats). This is more flexible way of enumerating data. 64K
>> namespace id is a very large namespace and can be reduced to 64 IANA
>> specified name spaces. Separate private name space can be allowed instead
>> of interleaving of opaque data in the IANA allocated name space as
>> suggested in the current draft “opaque state snapshot”.
>>
>> This is the most contentious of all. In the current proposal, as new data
>> fields are added, there is a corresponding trace type bit need in the
>> header. This essentially means that all possible data fields need to be
>> enumerated. Given that we there are 64K names spaces allowed, I don’t see
>> how we can fit all possible data fields in this 24bit vector. I know there
>> was a suggestion of keeping last bit as an extension bit but it is still
>> scalable and/or easy to implement in hardware. Besides this the data fields
>> are not annotated/encoded with the data type, something like in IPFIX
>> https://tools.ietf.org/html/rfc7011#section-6.1
>>
>>
>>
>> Feedback 4:
>>
>> There is no version field in the data header and this will make
>> interoperability challenging. Standard will evolve and headers bit
>> definition and/or trace type will change and without version field HW will
>> not be able to correctly handle the IOAM data headers.
>>
>>
>>
>> Feedback 5:
>>
>>    - GRE encaped IOAM packets will traverse a different network path
>>    then the original packet
>>    - Not all packets can be GRE encaped to avoid the previous problem,
>>    due to wastage of network bandwidth (typically sampled traffic is used for
>>    IOAM). What about native GRE traffic, will it get further encaped in
>>    another GRE tunnel and so forth.
>>    - IP header protocol will point to GRE IP proto and IOAM ethertype
>>    (pending allocation by IEEE) need to be read from the GRE header to detect
>>    an IOAM packet. This means parsing performance penalty for all regular GRE
>>    (non IOAM) traffic.
>>
>>
>>
>> Thanks,
>>
>> -Jai
>>
>>
>>
>

---

From: "Frank Brockners (fbrockne)" <[email-masked]>
Date: Thu, 22 Aug 2019 12:52:11 +0000

Hi Vijay,

note that you don’t necessarily need to “tunnel” – you can just use the GRE header to sequence-in IOAM.

Cheers, Frank

From: Vijay Rangarajan <[email-masked]>
Sent: Donnerstag, 22. August 2019 05:31
To: Carlos Pignataro (cpignata) <[email-masked]>
Cc: Jai Kumar <[email-masked]>; [email-masked]; IETF IPPM WG <[email-masked]>; Frank Brockners (fbrockne) <[email-masked]>; Hugh Holbrook <[email-masked]>; Anoop Ghanwani <[email-masked]>; OU, Heidi <[email-masked]>; Surendra Anubolu <[email-masked]>; John Lemon <[email-masked]>
Subject: Re: [ippm] Review on draft-ietf-ippm-ioam-data-06

Based on my understanding of the two drafts I have the following questions and concerns:

  1.  If I understand correctly, to deploy inband telemetry, we would need to construct GRE tunnels coinciding with the IOAM domain?
  2.  GRE typically requires configuration to provision the tunnels. Provisioning and managing these tunnels and keeping these updated as the network grows/shrinks could be a significant overhead.
  3.  In order to get the benefit of telemetry, we are imposing a change in forwarding protocol/topology and configuration - which, I feel is not desirable. For example, a customer might have basic L3 routing enabled and the expectation would be for inband telemetry to work seamlessly, without having to revamp the network with GRE tunnels and such. This could be a significant barrier to deployment.
  4.  If sampling is used to select packets for performing IOAM encap, is the expectation that only sampled IOAM packets go through GRE encap? Or all data packets?
  5.  Due to network nodes inserting the IOAM data, the inner L3/L4 headers keep getting pushed deeper. I would imagine this gets challenging for ASICs to access these fields for hashing/load balancing.
  6.  Assuming only a subset of packets in a flow are subject to IOAM (based on sampling), how do we ensure these packets take the same network path as the rest of the packets in the flow?
Thanks,
Vijay

On Wed, Aug 21, 2019 at 5:04 PM Carlos Pignataro (cpignata) <[email-masked]<mailto:[email-masked]>> wrote:
Hello, Vijay,

Thanks!

Thanks,
Vijay

Feedback 1:

Feedback 2:

Feedback 3:

Feedback 4:

Feedback 5:

Thanks,
-Jai

---

From: Vijay Rangarajan <[email-masked]>
Date: Thu, 22 Aug 2019 19:52:44 +0530

Hi Frank:
Thanks, I knew I was missing something.
So basically what you are saying is - let's say we have a UDP packet, we
are just going to stick in the GRE header and IOAM Header and Metadata
in-between the original IP and UDP headers?

So, the next protocol in the IOAM Header should indicate the L4 protocol -
i.e UDP/TCP?
Looking at https://datatracker.ietf.org/doc/draft-weis-ippm-ioam-eth/, it
actually defines the "Next protocol" in the IOAM header to be an ethertype
value?

Thanks,
Vijay

On Thu, Aug 22, 2019 at 6:22 PM Frank Brockners (fbrockne) <
[email-masked]> wrote:

> *From:* Vijay Rangarajan <[email-masked]>
> *Sent:* Donnerstag, 22. August 2019 05:31
> *To:* Carlos Pignataro (cpignata) <[email-masked]>
> *Cc:* Jai Kumar <[email-masked]>;
> [email-masked]; IETF IPPM WG <[email-masked]>; Frank
> Brockners (fbrockne) <[email-masked]>; Hugh Holbrook <
> [email-masked]>; Anoop Ghanwani <[email-masked]>; OU, Heidi
> <[email-masked]>; Surendra Anubolu <
> [email-masked]>; John Lemon <[email-masked]>
> *Subject:* Re: [ippm] Review on draft-ietf-ippm-ioam-data-06
>
>
>
>    1. If I understand correctly, to deploy inband telemetry, we would
>    need to construct GRE tunnels coinciding with the IOAM domain?
>    3. In order to get the benefit of telemetry, we are imposing a change
>    in forwarding protocol/topology and configuration - which, I feel is not
>    5. Due to network nodes inserting the IOAM data, the inner L3/L4
>    headers keep getting pushed deeper. I would imagine this gets challenging
>    for ASICs to access these fields for hashing/load balancing.
> Given that deployments are sensitive to the IOAM overhead (specially in 5G
> deployments), it’s a 50% fixed overhead savings on a per packet basis.
>
>
>
>
>
> Feedback 2:
>    - GRE encaped IOAM packets will traverse a different network path then
>    the original packet

---

From: "OU, Heidi" <[email-masked]>
Date: Fri, 23 Aug 2019 02:29:04 +0800

Hi Frank,

I also have a question on the encapsulation: If you can get a new ethertype for IOAM, why not insert IOAM data directly after layer2 MAC?  instead of adding a GRE header for IOAM.

Thanks
Heidi

From: Vijay Rangarajan <[email-masked]>
Date: Thursday, August 22, 2019 at 7:22 AM
To: "Frank Brockners (fbrockne)" <[email-masked]>
Cc: "Carlos Pignataro (cpignata)" <[email-masked]>, Jai Kumar <[email-masked]>, "[email-masked]" <[email-masked]>, IETF IPPM WG <[email-masked]>, Hugh Holbrook <[email-masked]>, Anoop Ghanwani <[email-masked]>, "OU, Heidi" <[email-masked]>, Surendra Anubolu <[email-masked]>, John Lemon <[email-masked]>
Subject: Re: [ippm] Review on draft-ietf-ippm-ioam-data-06

Hi Frank:

So basically what you are saying is - let's say we have a UDP packet, we are just going to stick in the GRE header and IOAM Header and Metadata in-between the original IP and UDP headers?

So, the next protocol in the IOAM Header should indicate the L4 protocol - i.e UDP/TCP? 

Looking at https://datatracker.ietf.org/doc/draft-weis-ippm-ioam-eth/, it actually defines the "Next protocol" in the IOAM header to be an ethertype value?

Thanks,

Vijay

On Thu, Aug 22, 2019 at 6:22 PM Frank Brockners (fbrockne) <[email-masked]> wrote:

Hi Vijay,

Cheers, Frank

If I understand correctly, to deploy inband telemetry, we would need to construct GRE tunnels coinciding with the IOAM domain?
GRE typically requires configuration to provision the tunnels. Provisioning and managing these tunnels and keeping these updated as the network grows/shrinks could be a significant overhead.
In order to get the benefit of telemetry, we are imposing a change in forwarding protocol/topology and configuration - which, I feel is not desirable. For example, a customer might have basic L3 routing enabled and the expectation would be for inband telemetry to work seamlessly, without having to revamp the network with GRE tunnels and such. This could be a significant barrier to deployment.
If sampling is used to select packets for performing IOAM encap, is the expectation that only sampled IOAM packets go through GRE encap? Or all data packets?
Due to network nodes inserting the IOAM data, the inner L3/L4 headers keep getting pushed deeper. I would imagine this gets challenging for ASICs to access these fields for hashing/load balancing.
Assuming only a subset of packets in a flow are subject to IOAM (based on sampling), how do we ensure these packets take the same network path as the rest of the packets in the flow?
Thanks,

Vijay

On Wed, Aug 21, 2019 at 5:04 PM Carlos Pignataro (cpignata) <[email-masked]> wrote:

Hello, Vijay, 

Thanks!

Hello all: 

Thanks,

Vijay

Hello Frank,

 

Feedback 1:

 

Feedback 2:

Feedback 3:

Feedback 4:

Feedback 5:

Thanks,

-Jai

---

From: Tom Herbert <[email-masked]>
Date: Thu, 22 Aug 2019 12:33:35 -0700

> I also have a question on the encapsulation: If you can get a new ethertype for IOAM, why not insert IOAM data directly after layer2 MAC?  instead of adding a GRE header for IOAM.
Because, we need a packet format that is compatible with existing
network devices. In light of that, GRE is more preferable than using
the new Ethertype directly in an Ethernet frame. There will also be
similar arguments made for using GRE/IP, and UDP encapsulation over
IP, and there was even a proposal to somehow insert the IOAM data
immediately after the TCP header and before the TCP data. All of these
are attempts to use protocol headers that are thought to be most
palatable to intermediate devices and maximize the chances of
efficient delivery.

IMO, all of the aforementioned techniques have some problem or aren't
clean (including the GRE solution). The best solution, and most
architecturally correct and generic one, is an IOAM option in
Hop-by-Hop extension headers.

Tom

> Section 4.2..1 Pre-allocated and Incremental Trace Options
>
> _______________________________________________
> ippm mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/ippm

---

From: "OU, Heidi" <[email-masked]>
Date: Fri, 23 Aug 2019 04:51:18 +0800

Tom,

Thanks for the reply. Anoop also gave some insight offline on the ASIC restriction.

About adding  IOAM as an IP option, I thought it had been  proposed in 
	https://tools.ietf.org/html/draft-kumar-ippm-ifa-01#page-10
What prevent us from doing that?

>From deployment point of view,  as long as the INT packet can go through the exact same path/queue as the  pre-encap'd packets, we are fine. 

Thanks
- Heidi

On 2019/8/22, 12:41 PM, "Tom Herbert" <[email-masked]> wrote:

    
    Tom

---

From: Tom Herbert <[email-masked]>
Date: Thu, 22 Aug 2019 14:11:28 -0700

> What prevent us from doing that?

Yeah, IP options would be obstensibly be an obvious choice, but proper
support in middleboxes is notoriously bad. Also, they're limited to
forty bytes. In reality, I doubt no one is seriously considering use
of new IPv4 options. However, IPv6 extension headers are an active
area of development (including the definition of IOAM options). I've
written a draft to allow IPv4 to carry the same HBH extension headers
in IPv4 to bridge the gap between v4 and v6.

Tom

---

From: Barak Gafni <[email-masked]>
Date: Thu, 22 Aug 2019 21:27:58 +0000

Hi,
You may be interested to take a look on a draft that discusses the use of the IP options for IOAM: https://tools.ietf.org/html/draft-gafni-ippm-ioam-ipv4-options-00
I do agree that IP options is an area we should use for this application. The good thing about IP options is that the header architecture enable the implementations to easily go over the options without the need to be aware and parse them, as a backwards compatibility consideration.

Thanks,
Barak

-----Original Message-----
From: ippm <[email-masked]> On Behalf Of Tom Herbert
Sent: Thursday, August 22, 2019 2:11 PM
To: OU, Heidi <[email-masked]>
Cc: [email-masked]; IETF IPPM WG <[email-masked]>; Hugh Holbrook <[email-masked]>; Anoop Ghanwani <[email-masked]>; Surendra Anubolu <[email-masked]>
Subject: Re: [ippm] Review on draft-ietf-ippm-ioam-data-06

On Thu, Aug 22, 2019 at 1:51 PM OU, Heidi <[email-masked]> wrote:
> https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Ftool
> s.ietf.org%2Fhtml%2Fdraft-kumar-ippm-ifa-01%23page-10&amp;data=02%7C01
> %7Cgbarak%40mellanox.com%7Ce4ccee89b9bc4e1274ce08d727455b2a%7Ca652971c
> 7d2e4d9ba6a4d149256f461b%7C0%7C0%7C637021051167634230&amp;sdata=gt0iYj
> 8NTD2UVgTVshJ011ndwG46tHRcEhXQ5NGvDAE%3D&amp;reserved=0

Yeah, IP options would be obstensibly be an obvious choice, but proper support in middleboxes is notoriously bad. Also, they're limited to forty bytes. In reality, I doubt no one is seriously considering use of new IPv4 options. However, IPv6 extension headers are an active area of development (including the definition of IOAM options). I've written a draft to allow IPv4 to carry the same HBH extension headers in IPv4 to bridge the gap between v4 and v6.

Tom

>     > Looking at https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdatatracker.ietf.org%2Fdoc%2Fdraft-weis-ippm-ioam-eth%2F&amp;data=02%7C01%7Cgbarak%40mellanox.com%7Ce4ccee89b9bc4e1274ce08d727455b2a%7Ca652971c7d2e4d9ba6a4d149256f461b%7C0%7C0%7C637021051167634230&amp;sdata=PykE1LWuF5gfIvL3ZiS242rAKnh%2FEZ5PXjZrp6dQXy4%3D&amp;reserved=0, it actually defines the "Next protocol" in the IOAM header to be an ethertype value?
>     >
>     >
>     >
>     > Thanks,
>     >
>     > Vijay
>     >
>     >
>     >
>     >
>     >
>     > Please see https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdatatracker.ietf.org%2Fdoc%2Fdraft-weis-ippm-ioam-eth%2F&amp;data=02%7C01%7Cgbarak%40mellanox.com%7Ce4ccee89b9bc4e1274ce08d727455b2a%7Ca652971c7d2e4d9ba6a4d149256f461b%7C0%7C0%7C637021051167634230&amp;sdata=PykE1LWuF5gfIvL3ZiS242rAKnh%2FEZ5PXjZrp6dQXy4%3D&amp;reserved=0, and the document this replaces.
>     >
>     >
>     >
>     > Thanks!
>     >
>     > https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Ftools.ietf.org%2Fhtml%2Frfc7011%23section-3.4&amp;data=02%7C01%7Cgbarak%40mellanox.com%7Ce4ccee89b9bc4e1274ce08d727455b2a%7Ca652971c7d2e4d9ba6a4d149256f461b%7C0%7C0%7C637021051167634230&amp;sdata=WgPKon9dcPr2bhKG0amAA2rLs9DVKbQTUmjF7FZMYHs%3D&amp;reserved=0
>     >
>     >
>     >
>     > Feedback 3:
>     >
>     > This is the most contentious of all. In the current proposal, as new data fields are added, there is a corresponding trace type bit need in the header. This essentially means that all possible data fields need to be enumerated. Given that we there are 64K names spaces allowed, I don’t see how we can fit all possible data fields in this 24bit vector. I know there was a suggestion of keeping last bit as an extension bit but it is still scalable and/or easy to implement in hardware. Besides this the data fields are not annotated/encoded with the data type, something like in IPFIX https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Ftools.ietf.org%2Fhtml%2Frfc7011%23section-6.1&amp;data=02%7C01%7Cgbarak%40mellanox.com%7Ce4ccee89b9bc4e1274ce08d727455b2a%7Ca652971c7d2e4d9ba6a4d149256f461b%7C0%7C0%7C637021051167644216&amp;sdata=1lWHooNEDwt4Rh8oxXy2VlyyRgoWBZr6Ig%2BYvQQJfYk%3D&amp;reserved=0
>     >
>     >
>     >
>     > Feedback 4:
>     >
> https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.
> ietf.org%2Fmailman%2Flistinfo%2Fippm&amp;data=02%7C01%7Cgbarak%40mella
> nox.com%7Ce4ccee89b9bc4e1274ce08d727455b2a%7Ca652971c7d2e4d9ba6a4d1492
> 56f461b%7C0%7C0%7C637021051167644216&amp;sdata=VrxNh%2FdRh%2FYasyWtFXS
> et2I5iADIY0U5vDjLd6ObAmU%3D&amp;reserved=0
>
>
>

_______________________________________________
ippm mailing list
[email-masked]
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.ietf.org%2Fmailman%2Flistinfo%2Fippm&amp;data=02%7C01%7Cgbarak%40mellanox.com%7Ce4ccee89b9bc4e1274ce08d727455b2a%7Ca652971c7d2e4d9ba6a4d149256f461b%7C0%7C0%7C637021051167644216&amp;sdata=VrxNh%2FdRh%2FYasyWtFXSet2I5iADIY0U5vDjLd6ObAmU%3D&amp;reserved=0

---

From: Anoop Ghanwani <[email-masked]>
Date: Thu, 22 Aug 2019 15:22:16 -0700

Hi Barak/Tom,

Using IP options would be cleaner but there are other issues that make it
undesirable.  Most existing silicon will either blindly
forward/discard/punt to CPU all packets with IP options.  If we pick the
forward option, then it opens up a hole that allows packets with any other
IP options through.  Additionally, when forwarding many will not parse
beyond the option header so things like ACLs and hashing for LAG/ECMP will
break.  Picking any of the other 2 options (discard/punt) means this
protocol is no longer backwards compatible.

I may be wrong, but I think it may be hard to convince the IETF to move
forward with a proposal that needs to extend IPv4 in any way, hence e.g.
segment routing is supported only with IPv6.

Thanks,
Anoop

On Thu, Aug 22, 2019 at 2:28 PM Barak Gafni <[email-masked]> wrote:

> Hi,
> You may be interested to take a look on a draft that discusses the use of
> the IP options for IOAM:
> https://tools.ietf.org/html/draft-gafni-ippm-ioam-ipv4-options-00
> I do agree that IP options is an area we should use for this application.
> The good thing about IP options is that the header architecture enable the
> implementations to easily go over the options without the need to be aware
> and parse them, as a backwards compatibility consideration.
>
> Thanks,
> Barak
>
> Cc: [email-masked]; IETF IPPM WG <[email-masked]>;
> Hugh Holbrook <[email-masked]>; Anoop Ghanwani <
> [email-masked]>; Surendra Anubolu <[email-masked]>
> support in middleboxes is notoriously bad. Also, they're limited to forty
> bytes. In reality, I doubt no one is seriously considering use of new IPv4
> options. However, IPv6 extension headers are an active area of development
> (including the definition of IOAM options). I've written a draft to allow
> IPv4 to carry the same HBH extension headers in IPv4 to bridge the gap
> between v4 and v6.
>
> Tom
>
> >
> > From deployment point of view,  as long as the INT packet can go through
> the exact same path/queue as the  pre-encap'd packets, we are fine.
> >
> > Thanks
> > - Heidi
> >
> >     > I also have a question on the encapsulation: If you can get a new
> ethertype for IOAM, why not insert IOAM data directly after layer2 MAC?
> instead of adding a GRE header for IOAM.
> >     >
> >     > Cc: "Carlos Pignataro (cpignata)" <[email-masked]>, Jai Kumar
> <[email-masked]>, "[email-masked]" <
> [email-masked]>, IETF IPPM WG <[email-masked]>, Hugh
> Holbrook <[email-masked]>, Anoop Ghanwani <[email-masked]>,
> "OU, Heidi" <[email-masked]>, Surendra Anubolu <
> [email-masked]>, John Lemon <[email-masked]>
> >     > So basically what you are saying is - let's say we have a UDP
> packet, we are just going to stick in the GRE header and IOAM Header and
> Metadata in-between the original IP and UDP headers?
> >     >
> >     >
> >     >
> >     > Looking at
> https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdatatracker.ietf.org%2Fdoc%2Fdraft-weis-ippm-ioam-eth%2F&amp;data=02%7C01%7Cgbarak%40mellanox.com%7Ce4ccee89b9bc4e1274ce08d727455b2a%7Ca652971c7d2e4d9ba6a4d149256f461b%7C0%7C0%7C637021051167634230&amp;sdata=PykE1LWuF5gfIvL3ZiS242rAKnh%2FEZ5PXjZrp6dQXy4%3D&amp;reserved=0,
> it actually defines the "Next protocol" in the IOAM header to be an
> ethertype value?
> >     >
> >     >
> >     >
> >     > Thanks,
> >     >
> >     > Vijay
> >     >
> >     >
> >     >
> >     >
> >     >
> >     > Cc: Jai Kumar <[email-masked]>;
> >     > If I understand correctly, to deploy inband telemetry, we would
> >     > GRE typically requires configuration to provision the tunnels.
> >     > In order to get the benefit of telemetry, we are imposing a change
> >     > If sampling is used to select packets for performing IOAM encap,
> is the expectation that only sampled IOAM packets go through GRE encap? Or
> all data packets?
> >     > Due to network nodes inserting the IOAM data, the inner L3/L4
> >     > Assuming only a subset of packets in a flow are subject to IOAM
> >     > Please see
> https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdatatracker.ietf.org%2Fdoc%2Fdraft-weis-ippm-ioam-eth%2F&amp;data=02%7C01%7Cgbarak%40mellanox.com%7Ce4ccee89b9bc4e1274ce08d727455b2a%7Ca652971c7d2e4d9ba6a4d149256f461b%7C0%7C0%7C637021051167634230&amp;sdata=PykE1LWuF5gfIvL3ZiS242rAKnh%2FEZ5PXjZrp6dQXy4%3D&amp;reserved=0,
> and the document this replaces.
> >     >
> >     >
> >     >
> >     > Thanks!
> >     >
> >     > In reading "draft-ietf-ippm-ioam-data-06", I don't see mention of
> GRE encap. The draft, in fact in Sec 3, says the following - "The in-situ
> OAM data field can be transported by a variety of transport protocols,
> >     > Is there another document, or a description somewhere, that talks
> about how IOAM is proposed to be carried in GRE? what would be the GRE
> payload, the GRE protocol type etc?
> >     >
> >     >
> >     >
> >     > Thanks,
> >     >
> >     > Vijay
> >     >
> >     >
> >     >
> >     >
> >     >
> >     > This is in context of our conversation at IETF105. My goal is to
> provide input and improve current IOAM data draft with the learnings we had
> with IFA deployment.
> >     >
> >     > This feedback is based on various customer interactions and
> concerns raised by them wrt IOAM. Each feedback is a longer topic and I am
> starting this thread as a summary email. This is just highlighting the
> issues and not yet proposing any solution.
> >     >
> >     >
> >     >
> >     >
> >     >
> >     > Feedback 1:
> >     >
> >     > Given that deployments are sensitive to the IOAM overhead
> (specially in 5G deployments), it’s a 50% fixed overhead savings on a per
> packet basis.
> >     >
> >     >
> >     >
> >     >
> >     >
> >     > Feedback 2:
> >     >
> >     > Namespaces should be treated as templates (similar to IPFIX
> template record formats). This is more flexible way of enumerating data.
> 64K namespace id is a very large namespace and can be reduced to 64 IANA
> >     > This is the most contentious of all. In the current proposal, as
> new data fields are added, there is a corresponding trace type bit need in
> the header. This essentially means that all possible data fields need to be
> https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Ftools.ietf.org%2Fhtml%2Frfc7011%23section-6.1&amp;data=02%7C01%7Cgbarak%40mellanox.com%7Ce4ccee89b9bc4e1274ce08d727455b2a%7Ca652971c7d2e4d9ba6a4d149256f461b%7C0%7C0%7C637021051167644216&amp;sdata=1lWHooNEDwt4Rh8oxXy2VlyyRgoWBZr6Ig%2BYvQQJfYk%3D&amp;reserved=0
> >     >
> >     >
> >     >
> >     > Feedback 4:
> >     >
> >     > Handling of TCP/UDP traffic using GRE encap is not acceptable.
> Here are some of the issues I can think of
> >     >
> >     > Not all packets can be GRE encaped to avoid the previous problem,
> >     > IP header protocol will point to GRE IP proto and IOAM ethertype

---

From: Tom Herbert <[email-masked]>
Date: Thu, 22 Aug 2019 15:45:03 -0700

On Thu, Aug 22, 2019 at 3:22 PM Anoop Ghanwani <[email-masked]> wrote:
>
> Hi Barak/Tom,
>
> Using IP options would be cleaner but there are other issues that make it undesirable.  Most existing silicon will either blindly forward/discard/punt to CPU all packets with IP options.  If we pick the forward option, then it opens up a hole that allows packets with any other IP options through.  Additionally, when forwarding many will not parse beyond the option header so things like ACLs and hashing for LAG/ECMP will break.  Picking any of the other 2 options (discard/punt) means this protocol is no longer backwards compatible.

The problems are summarized in the famous "IP Options are not an
option" paper. https://www2.eecs.berkeley.edu/Pubs/TechRpts/2005/EECS-2005-24.html
>
> I may be wrong, but I think it may be hard to convince the IETF to move forward with a proposal that needs to extend IPv4 in any way, hence e.g. segment routing is supported only with IPv6.

Yes, that concern was raised. I think our argument would be something
like we're enabling an already defined protocol number to be used with
IPv4, we're not really extending or improving IPv4 protocol (Brian
Trammell's idea). The next draft on the IPv4 EH will highlight this
and narrow the use cases that were in last draft.

Tom

---

From: Greg Mirsky <[email-masked]>
Date: Thu, 22 Aug 2019 16:41:10 -0700

Hi Tom,
I think that the right solution to the problem of collecting telemetry
information as experienced by a data packet is to disconnect origination of
such information and transporting it. An iOAM packet may be used as a
trigger to collect the required data on a node. But the same packet doesn't
have to transport that information. That could be done either using
Postcard-based
Telemetry
<https://datatracker.ietf.org/doc/draft-song-ippm-postcard-based-telemetry/>
or Hybrid Two-Step
<https://datatracker.ietf.org/doc/draft-mirsky-ippm-hybrid-two-step/>methods.
In that case, IP options provide sufficient space to encode the profile of
telemetry data to collect on a node.

Regards,
Greg

On Thu, Aug 22, 2019 at 12:34 PM Tom Herbert <[email-masked]> wrote:

> > Cc: "Carlos Pignataro (cpignata)" <[email-masked]>, Jai Kumar <
> [email-masked]>, "[email-masked]" <
> > Looking at https://datatracker.ietf.org/doc/draft-weis-ippm-ioam-eth/,
> > If I understand correctly, to deploy inband telemetry, we would need to
> construct GRE tunnels coinciding with the IOAM domain?
> > In order to get the benefit of telemetry, we are imposing a change in
> > If sampling is used to select packets for performing IOAM encap, is the
> expectation that only sampled IOAM packets go through GRE encap? Or all
> data packets?
> > Due to network nodes inserting the IOAM data, the inner L3/L4 headers
> > Assuming only a subset of packets in a flow are subject to IOAM (based
> on sampling), how do we ensure these packets take the same network path as
> the rest of the packets in the flow?
> >
> > Thanks,
> >
> > Vijay
> >
> >
> >
> >
> >
> > This is the most contentious of all. In the current proposal, as new
> data fields are added, there is a corresponding trace type bit need in the
> > Not all packets can be GRE encaped to avoid the previous problem, due to
> wastage of network bandwidth (typically sampled traffic is used for IOAM).
> What about native GRE traffic, will it get further encaped in another GRE
> tunnel and so forth.

---

From: Jai Kumar <[email-masked]>
Date: Thu, 22 Aug 2019 17:00:56 -0700

Hi Greg,

Problem with the postcard approach in MSDC would be the pps rate collector need to handle. Specially spine nodes will generate a flood of postcards. It becomes cost prohibitive to deploy such approach.

Keeping the data in the packet itself and managing it on the leaf nodes to generate reports helps scale, where multiple hops and packet information is coalesced in a single report.

Thanks,

-Jai

From: ippm <[email-masked]> on behalf of Greg Mirsky <[email-masked]>
Date: Thursday, August 22, 2019 at 4:41 PM
To: Tom Herbert <[email-masked]>
Cc: "[email-masked]" <[email-masked]>, IETF IPPM WG <[email-masked]>, Hugh Holbrook <[email-masked]>, Anoop Ghanwani <[email-masked]>, "OU, Heidi" <[email-masked]>, Surendra Anubolu <[email-masked]>
Subject: Re: [ippm] Review on draft-ietf-ippm-ioam-data-06

Hi Tom,

I think that the right solution to the problem of collecting telemetry information as experienced by a data packet is to disconnect origination of such information and transporting it. An iOAM packet may be used as a trigger to collect the required data on a node. But the same packet doesn't have to transport that information. That could be done either using Postcard-based Telemetry or Hybrid Two-Step methods. In that case, IP options provide sufficient space to encode the profile of telemetry data to collect on a node.

Regards,

Greg

Tom

_______________________________________________ ippm mailing list [email-masked] https://www.ietf.org/mailman/listinfo/ippm

---

From: Greg Mirsky <[email-masked]>
Date: Thu, 22 Aug 2019 17:08:40 -0700

Hi Jai,
PBT does have that issue, agree, and is better suited in other scenarios.
Then why not use HTS for MSDC? But first, we discuss that generation of
information and transport don't have to be done with the same packet. In
other words, we separate the production of telemetry information from the
collection and transport of it.

Regards,
Greg

On Thu, Aug 22, 2019 at 5:00 PM Jai Kumar <[email-masked]> wrote:

> Hi Greg,
>
>
>
> Problem with the postcard approach in MSDC would be the pps rate collector
> need to handle. Specially spine nodes will generate a flood of postcards.
> It becomes cost prohibitive to deploy such approach.
>
> Keeping the data in the packet itself and managing it on the leaf nodes to
> generate reports helps scale, where multiple hops and packet information is
> coalesced in a single report.
>
>
>
> Thanks,
>
> -Jai
>
>
>
> *From: *ippm <[email-masked]> on behalf of Greg Mirsky <
> [email-masked]>
> *Date: *Thursday, August 22, 2019 at 4:41 PM
> *To: *Tom Herbert <[email-masked]>
> *Cc: *"[email-masked]" <
> [email-masked]>
> *Subject: *Re: [ippm] Review on draft-ietf-ippm-ioam-data-06
>
>
>
> Hi Tom,
>
> have to transport that information. That could be done either using Postcard-based
> Telemetry
> _______________________________________________ ippm mailing list
> [email-masked] https://www.ietf.org/mailman/listinfo/ippm
>

---

From: Tom Herbert <[email-masked]>
Date: Thu, 22 Aug 2019 17:09:02 -0700

> Problem with the postcard approach in MSDC would be the pps rate collector need to handle. Specially spine nodes will generate a flood of postcards. It becomes cost prohibitive to deploy such appro…
And that also sounds like the basis for an amplification DOS attack. A
similar concern is also raised for loopback flag in IOAM.

Tom

---

From: Tianran Zhou <[email-masked]>
Date: Fri, 23 Aug 2019 05:44:44 +0000

Hi Jai,

Without any optimization, PBT will generate more packets. But all existing in-situ flow information telemetry methods will generate large amount of data and pps. No big difference.

The IFIT framework introduced how to deploy these techniques (IOAM, PBT, IFA,…) based on our experience in several carrier networks.
https://datatracker.ietf.org/doc/draft-song-opsawg-ifit-framework/

In general, the smart flow selection and export data reduction will reduce the pps problem. We have tested many use cases with our customers and get very good feedback.

Tianran

From: ippm [mailto:[email-masked]] On Behalf Of Jai Kumar
Sent: Friday, August 23, 2019 8:01 AM
To: Greg Mirsky <[email-masked]>; Tom Herbert <[email-masked]>
Cc: [email-masked]; IETF IPPM WG <[email-masked]>; Hugh Holbrook <[email-masked]>; Anoop Ghanwani <[email-masked]>; OU, Heidi <[email-masked]>; Surendra Anubolu <[email-masked]>
Subject: Re: [ippm] Review on draft-ietf-ippm-ioam-data-06

Hi Greg,

Thanks,
-Jai

From: ippm <[email-masked]<mailto:[email-masked]>> on behalf of Greg Mirsky <[email-masked]<mailto:[email-masked]>>
Date: Thursday, August 22, 2019 at 4:41 PM
To: Tom Herbert <[email-masked]<mailto:[email-masked]>>
Cc: "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>, IETF IPPM WG <[email-masked]<mailto:[email-masked]>>, Hugh Holbrook <[email-masked]<mailto:[email-masked]>>, Anoop Ghanwani <[email-masked]<mailto:[email-masked]>>, "OU, Heidi" <[email-masked]<mailto:[email-masked]>>, Surendra Anubolu <[email-masked]<mailto:[email-masked]>>
Subject: Re: [ippm] Review on draft-ietf-ippm-ioam-data-06

Hi Tom,
I think that the right solution to the problem of collecting telemetry information as experienced by a data packet is to disconnect origination of such information and transporting it. An iOAM packet may be used as a trigger to collect the required data on a node. But the same packet doesn't have to transport that information. That could be done either using Postcard-based Telemetry<https://datatracker.ietf.org/doc/draft-song-ippm-postcard-based-telemetry/> or Hybrid Two-Step <https://datatracker.ietf.org/doc/draft-mirsky-ippm-hybrid-two-step/> methods. In that case, IP options provide sufficient space to encode the profile of telemetry data to collect on a node.

Regards,
Greg

On Thu, Aug 22, 2019 at 12:34 PM Tom Herbert <[email-masked]<mailto:[email-masked]>> wrote:

Tom

>
>
> Thanks
>
> Heidi
>
>
>
> From: Vijay Rangarajan <[email-masked]<mailto:[email-masked]>>
> To: "Frank Brockners (fbrockne)" <fbrockne@cisco..com<mailto:[email-masked]>>
> Cc: "Carlos Pignataro (cpignata)" <[email-masked]<mailto:[email-masked]>>, Jai Kumar <[email-masked]<mailto:[email-masked]>>, "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>, IETF IPPM WG <[email-masked]<mailto:[email-masked]>>, Hugh Holbrook <[email-masked]<mailto:[email-masked]>>, Anoop Ghanwani <[email-masked]<mailto:[email-masked]>>, "OU, Heidi" <[email-masked]<mailto:[email-masked]>>, Surendra Anubolu <[email-masked]<mailto:[email-masked]>>, John Lemon <[email-masked]<mailto:[email-masked]>>
> On Thu, Aug 22, 2019 at 6:22 PM Frank Brockners (fbrockne) <[email-masked]<mailto:[email-masked]>> wrote:
>
> Hi Vijay,
>
>
>
> From: Vijay Rangarajan <[email-masked]<mailto:[email-masked]>>
> To: Carlos Pignataro (cpignata) <[email-masked]<mailto:[email-masked]>>
> Cc: Jai Kumar <[email-masked]<mailto:[email-masked]>>; [email-masked]<mailto:[email-masked]>; IETF IPPM WG <ippm@ietf..org<mailto:[email-masked]>>; Frank Brockners (fbrockne) <[email-masked]<mailto:[email-masked]>>; Hugh Holbrook <[email-masked]<mailto:[email-masked]>>; Anoop Ghanwani <[email-masked]<mailto:[email-masked]>>; OU, Heidi <[email-masked]<mailto:[email-masked]>>; Surendra Anubolu <[email-masked]<mailto:[email-masked]>>; John Lemon <[email-masked]<mailto:[email-masked]>>
> Please see https://datatracker.ietf.org/doc/draft-weis-ippm-ioam-eth/, and the document this replaces..
>
>
>
> Thanks!
>
> [email-masked]<mailto:[email-masked]>

_______________________________________________
ippm mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/ippm
_______________________________________________ ippm mailing list [email-masked]<mailto:[email-masked]> https://www.ietf.org/mailman/listinfo/ippm

---

From: Tianran Zhou <[email-masked]>
Date: Fri, 23 Aug 2019 05:49:48 +0000

Hi Tom,

As in my reply to Jai, in-situ flow information telemetry is not likely to export meta data in the per packet way. We can use smart flow selection and export data reduction as introduced in the IFIT framework to optimize.
https://datatracker.ietf.org/doc/draft-song-opsawg-ifit-framework/

Tianran

> From: ippm [mailto:[email-masked]] On Behalf Of Tom Herbert
> Sent: Friday, August 23, 2019 8:09 AM
> To: Jai Kumar <[email-masked]>
> Cc: [email-masked]; IETF IPPM WG <[email-masked]>; Hugh
> Holbrook <[email-masked]>; Anoop Ghanwani <[email-masked]>;
> OU, Heidi <[email-masked]>; Surendra Anubolu
> <[email-masked]>
> And that also sounds like the basis for an amplification DOS attack. A similar
> concern is also raised for loopback flag in IOAM.
> 
> Tom
> 
> > Keeping the data in the packet itself and managing it on the leaf nodes
> to generate reports helps scale, where multiple hops and packet information
> is coalesced in a single report.
> >
> >
> >
> > Thanks,
> >
> > -Jai
> >
> >
> >
> > Cc: "[email-masked]"
> > <[email-masked]>, IETF IPPM WG <[email-masked]>,
> > Hugh Holbrook <[email-masked]>, Anoop Ghanwani
> > <[email-masked]>, "OU, Heidi" <[email-masked]>,
> > Surendra Anubolu <[email-masked]>
> such information and transporting it. An iOAM packet may be used as a trigger
> to collect the required data on a node. But the same packet doesn't have to
> transport that information. That could be done either using Postcard-based
> Telemetry or Hybrid Two-Step methods. In that case, IP options provide
> sufficient space to encode the profile of telemetry data to collect on a node.
> >
> >
> >
> > Regards,
> >
> > Greg
> >
> >
> >
> > > I also have a question on the encapsulation: If you can get a new ethertype
> for IOAM, why not insert IOAM data directly after layer2 MAC?  instead of
> adding a GRE header for IOAM.
> > >
> > > <[email-masked]>, "[email-masked]"
> > > <[email-masked]>, IETF IPPM WG <[email-masked]>,
> > > Hugh Holbrook <[email-masked]>, Anoop Ghanwani
> > > <[email-masked]>, "OU, Heidi" <[email-masked]>,
> > > Surendra Anubolu <[email-masked]>, John Lemon
> > > <[email-masked]>
> > > So basically what you are saying is - let's say we have a UDP packet,
> we are just going to stick in the GRE header and IOAM Header and Metadata
> it actually defines the "Next protocol" in the IOAM header to be an ethertype
> value?
> > >
> > >
> > >
> > > Thanks,
> > >
> > > Vijay
> > >
> > >
> > >
> > >
> > >
> > > [email-masked]; IETF IPPM WG <[email-masked]>;
> > > Frank Brockners (fbrockne) <[email-masked]>; Hugh Holbrook
> > > <[email-masked]>; Anoop Ghanwani <[email-masked]>; OU,
> > > Heidi <[email-masked]>; Surendra Anubolu
> > > <[email-masked]>; John Lemon
> > > <[email-masked]>
> expectation that only sampled IOAM packets go through GRE encap? Or all data
> packets?
> data field can be transported by a variety of transport protocols, including
> NSH, Segment Routing, Geneve, IPv6, or IPv4.  Specification details for these
> different transport protocols are outside the scope of this document."
> > >
> > >
> > >
> input and improve current IOAM data draft with the learnings we had with IFA
> deployment.
> > >
> this thread as a summary email. This is just highlighting the issues and not
> yet proposing any solution.
> > >
> > >
> > >
> > >
> > >
> > > Feedback 1:
> > >
> record formats). This is more flexible way of enumerating data. 64K namespace
> id is a very large namespace and can be reduced to 64 IANA specified name
> spaces. Separate private name space can be allowed instead of interleaving
> of opaque data in the IANA allocated name space as suggested in the current
> draft “opaque state snapshot”.
> > >
> > > data fields are added, there is a corresponding trace type bit need
> > > in the header. This essentially means that all possible data fields
> > > need to be enumerated. Given that we there are 64K names spaces
> > > allowed, I don’t see how we can fit all possible data fields in this
> > > 24bit vector. I know there was a suggestion of keeping last bit as
> > > an extension bit but it is still scalable and/or easy to implement
> > > in hardware. Besides this the data fields are not annotated/encoded
> > > with the data type, something like in IPFIX
> > > GRE encaped IOAM packets will traverse a different network path then
> > > the original packet Not all packets can be GRE encaped to avoid the previous
> problem, due to wastage of network bandwidth (typically sampled traffic is
> used for IOAM). What about native GRE traffic, will it get further encaped
> in another GRE tunnel and so forth.
> > > IP header protocol will point to GRE IP proto and IOAM ethertype (pending
> allocation by IEEE) need to be read from the GRE header to detect an IOAM
> packet. This means parsing performance penalty for all regular GRE (non IOAM)
> traffic.
> > >
> > >
> > >
> > > Thanks,
> > >
> > > -Jai
> > >
> > >
> > >

---

From: Anoop Ghanwani <[email-masked]>
Date: Fri, 23 Aug 2019 15:02:23 -0700

Hi Frank,

As with Vijay, I am also interested in understanding the use of GRE for
in-sequencing OAM.  Do we end up needing an Ethertype for TCP/UDP?

Thanks,
Anoop

On Thu, Aug 22, 2019 at 10:06 AM Vijay Rangarajan <vijayr=
[email-masked]> wrote:

>> *To:* Carlos Pignataro (cpignata) <[email-masked]
>> <cpignata@cisco..com>>
>> [email-masked]>; Anoop Ghanwani <[email-masked]>; OU,
>> Heidi <heidi.ou@alibaba-inc..com <[email-masked]>>; Surendra
>> Anubolu <[email-masked]>; John Lemon <
>> [email-masked]>
>>    4. If sampling is used to select packets for performing IOAM encap,

---

From: Jai Kumar <[email-masked]>
Date: Fri, 23 Aug 2019 15:27:44 -0700

Hi Frank,

While we are trying to close on GRE based details please provide your thoughts on trace type and reducing the size of trace header as well.

Thanks,
-Jai

> On Aug 23, 2019, at 3:02 PM, Anoop Ghanwani <[email-masked]> wrote:
> 
> Hi Frank,
> 
> As with Vijay, I am also interested in understanding the use of GRE for in-sequencing OAM.  Do we end up needing an Ethertype for TCP/UDP?
> 
> Thanks,
> Anoop
> 
>> On Thu, Aug 22, 2019 at 10:06 AM Vijay Rangarajan <vijayr=[email-masked]> wrote:
>> Hi Frank:
>>> Cc: Jai Kumar <[email-masked]>; [email-masked]; IETF IPPM WG <[email-masked]>; Frank Brockners (fbrockne) <[email-masked]>; Hugh Holbrook <[email-masked]>; Anoop Ghanwani <[email-masked]>; OU, Heidi <heidi.ou@alibaba-inc...com>; Surendra Anubolu <[email-masked]>; John Lemon <[email-masked]>

---

From: "Carlos Pignataro (cpignata)" <[email-masked]>
Date: Sun, 25 Aug 2019 20:50:28 +0000

> On Aug 22, 2019, at 7:41 PM, Greg Mirsky <[email-masked]> wrote:
> 
> Hi Tom,

It is not about the size of the option.

Please read, even using NOOP:
http://conferences.sigcomm.org/imc/2004/papers/p336-medina.pdf
https://www2.eecs.berkeley.edu/Pubs/TechRpts/2005/EECS-2005-24.html
http://www.employees.org/~dwing/ipoptions/

-- Carlos.

>> wastage of network bandwidth (typically sampled traffic is used for IOAM)