Subject: [sfc] Metadata representation and transport

From: Nicolas BOUTHORS <[email-masked]>
Date: Thu, 10 Jul 2014 18:15:59 +0000

Hello,

Metadata transport is a feature provided by SFC that needs some additional thoughts as well.

B Rijsman has done a great job showing various option for Metadata sharing. Still there are 2 aspects that I would like to address further

- Metadata representation

- Metadata transport reliability

We will probably post a draft on the subject, but this can be done after Toronto has started. So for those
interested to provide some feedback, here are some highlights.

1. Problem Space
 Metadata representation
Metadata can be extremely varied in term of usage and content. It can represent the result of Deep Packet Inspection (DPI) performed on the traffic for example by the Classifier Service Function at the ingress of the Service Function Chain. It can contain information collected about the end user such as a policy identifier, a category or even represent an event related to the end-user session.
This information can be used by Service Functions on the chain, as well as by monitoring entities responsible to track usage for example in the Network Function Virtualization environment to feed a VNF manager or an Orchestrator.
Metadata being information shared by many network entities needs some means to be represented it in all of its dimensions: type definition, encoding and scope.
 Metadata transport service
As expected from service chain proposals, both NSH and SCH proposals define some means to carry metadata between Service Functions in a Chain. They can be classified as follow:

Dataplane Metadata:
are defined in the Service Function Chaining Problem Statement document. They are not considered to be part of the forwarding information of the SFC header. However they are expected to carry the result of antecedent classification, allowing Service Functions to take local policy decisions based on their values.

As such, they could also be interpreted directly by Service Function Forwarder to steer traffic to various Service Functions.
Offline Metadata:
Beyond Dataplane Metadata, Offline Metadata can be shared between Service Functions in a chain, using out of band, congruent or not, or hybrid models described in [RIJSMAN]

The hybrid model for example, defined in [RIJSMAN], utilizes the SFC header to transport some key values for correlation purposes. These Correlation Ids can be used by the Service Functions to recover the full associated contextual information.

Metadata, directly or indirectly, are transported hop by hop along a chain, in association to end-user traffic, the data payload of the SFC packets.
How these metadata are transported over a chain matters. Passing metadata directly or indirectly along packets is a service that must be analyzed from a reliability point of view.

Reliability requirements may vary depending on the nature of the metadata transported. Past experience for example in Mobile Network and data center with AAA Radius have shown that contextual information replication to various service function was indeed sensitive to packet loss events, and that adhoc solutions had to be implemented to detect them.
End to end TCP retransmission of packet lost does not ensure that associated Metadata will be reinserted in retransmitted packet. In addition, Event triggered metadata may have to be sent immediately on a chain even though no end-user traffic is being transmitted
Not all metadata needs a reliable transport. Repeated inline metadata can be used to cover several use cases, and some metadata loss can be acceptable.
Still, a reliable transport service for Metadata in SFC is expected. To this effect, an implementation is suggested in Section 3.3
2. Metadata representation
Metadata definition is that it provides contextual information about the data packets which traverse a Service Function Chain. This must be understood broadly.
Metadata can contain the result of traffic classification by Deep Packet Inspection (DPI). For example as an Application Id information which is tied to a traffic flow. There could be multiple flows with different application ids, in a chain.
Metadata can also contain the result of DPI data extraction, such as identify requested URL in HTTP. Such information can be passed to certain SF down the chain such as a URL filtering function.
Metadata can contain some punctual event information collected at the Ingress point of the chain and expected to be passed to all elements in the chain. Here this information may be triggered externally and generated only once, and be related to the tenant or the subscriber.
Metadata representation involves the definition of a set of information elements types and the encoding rules for their values.
Metadata representation can sometimes be performed by a single individual field with associated type and format. However, it is not always enough.

  *   Metadata may need multiple fields transported together to represented their values.
  *   Some addition fields may be required to describe the scope of the metadata itself. This can be any information element allowing to define the context of the associated metadata value. For example a throughput metadata field can have a port number and a switch address as its Scope information.

We explore these two axis: encoding and scope.
We propose IPFIX as a preferred means to represent Metadata in Service Chain messages for Out-of-band, Congruent or not; Metadata sharing.
2.1. Metadata Representation Requirements
Mandatatory Dataplane Metadata is always part of the SFC header, it is thus reasonable to consider that its representation scheme will be implicit: based on what the SFC protocol will dictate, their position in the SFC header is sufficient for the receiving end to infer their type and encoding scheme. For example, Context Header Fields in NSH are 32 bit fields.
However, it will not be the case for all metadata transported. Offline metadata, including congruent out-of-band metadata still need to be represented explicitly. This section addresses their specific case.
2.1.1. Metadata Encoding requirements
These requirements are applicable to out-of-band metadata (Congruent or not). It could be applicable with SCH on optional in-line metadata fields.
For interoperability purposes, metadata encoding MUST allow the receiving entity to identify the type and value of the information received as metadata
Metadata encoding MUST allow for encoding techniques supporting well known types and fields as well as proprietary extensions.
A receiving entity MUST be able to identify when incoming metadata type is unknown and MUST have a defined default action to handle it.
A piece of information may need multiple fields to be described. For example a tenant id and an ip address can be used to identify an server in a data center uniquely.
These groups of information have to be exchanged collectively, in a single message. In this case, a sending entity MUST specify that it is sending a set of metadata in a message.
This set of transported metadata elements MUST be specified under the form of a metadata template document uniquely defined for the chain.
A receiving entity MUST be able to detect if an incoming message contains its expected set of metadata elements.
2.1.2. Metadata Scope requirement
A piece of information may have to be qualified by some attributes allowing to identify its particular scope. For example in a Gi environment, the radio type in use may be used as a scoping criteria for a jitter or latency measurement in a video traffic transported in a particular service chaing.
Scope can apply to some individual metadata elements or to a set of metadata elements. How a scope applies to a set of transported metadata elements should be defined by a specification under the form of a metadata template document uniquely identified for the chain.
2.2. IPFIX Metadata representation
So far, unordered sequences of Type Length Value encoded fields have been proposed to transport metadata. It is not clear how structured types are supported, and no distinction is done between the metadata values and their scoping values. Although the SCH proposal provides an optional 24-bit Organizational Unique Identifier, there is no namespace mechanism allowing to separate type definition spaces per Tenants or per chain.
We suggest to leverage the work done by IETF on similar subject.
A natural candidate to leverage is IPFIX [RFC7011]: IPFIX is a means for transmitting Traffic Flow information over the network. In order to transmit Traffic Flow information, it provides a common representation of flow data and a standard means of communicating them.
Metadata collected by Network Node and Service Node SHOULD be encoded in template following the principles described in IPFIX[RFC7011].
An IPFIX Message consisting of interleaved Template, Data, and Options Template Sets, as shown in Figure 1. Here, Template and Options Template Sets , which are optional, are shown.

   +--------+--------------------------------------------------------+
   |        | +----------+ +---------+     +-----------+ +---------+ |
   |Message | | Template | | Data    |     | Options   | | Data    | |
   | Header | | Set      | | Set     | ... | Template  | | Set     | |
   |        | |          | |         |     | Set       | |         | |
   |        | +----------+ +---------+     +-----------+ +---------+ |
   +--------+--------------------------------------------------------+

Figure 1: IPFIX Message Format
The Template Set describes the data transmitted in the following Data Set. It is an optional component of the message. The value of the metadata is encoded in the first Data Set. This Data Set contains a template Id field as a reference to its defining Template Set.
The Options Template Set describes the data to be transmitted as scope information. It is an optional component of the message. The value of the scope information is encoded in the second Data Set element. It no scope information is present, then only the first Data Set is present in the message.
The Option Template Set and following Data Set are used to describe the scope of the metadata transmitted. For example, the metadata collected is relevant to a PDP Context or a particular line card of a particular switch.

2.2.4.3. Example:
The Metadata Exporting Process creates a Template Record with a few Information Elements: amongst other things, the Application Id. For example:

        - sourceIPv4Address (key field)
        - destinationIPv4Address (key field)
        - protocol (key field)
        - destinationTransportPort (key field)
        - applicationId (key field)
        - octetTotalCount (non key field)

        For example, a Flow Record corresponding to the above
        Template Record may contain:

            { sourceIPv4Address=192.0.2.1,
              destinationIPv4Address=192.0.2.2,
              protocol=17,
              destinationTransportPort=23,
              applicationId='3..80',
              octetTotalCount=123456 }

The Options Data Record associated with the examples above would contain the Scoping inforamtion:

     Scope:
             - servicePath,
             - serviceIndex,
             - applicationId,
             - applicationName
             - applicationDescription.

      For example:

            {
              servicePath=0x000b,
              serviceIndex=0x000c,
              applicationId='13...10000',
              applicationName="webex",
              applicationDescription="Webex application" }

Scope information is useful when sending metadata offline, as it can contain information related to the chain and possibly to the flow for which this metadata record is relevant. Here servicePath and serviceIndex are thus included in the Template.
2.2.4.4. IPFIX encoding and template provisioning:
IPFIX is a quite compact encoding
For a template defined as followed and shared by the SF in a chain.
IPFIX template record shared by SF:
Note that an XML representation of IPFIX template record could be defined and used to provision Service Functions.

 0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |         Set ID = 2            |      Length = 28 octets       |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |       Template ID 256         |       Field Count = 5         |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |0|    sourceIPv4Address = 8    |       Field Length = 4        |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |0| destinationIPv4Address = 12 |       Field Length = 4        |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |0|  ipNextHopIPv4Address = 15  |       Field Length = 4        |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |0|    packetDeltaCount = 2     |       Field Length = 4        |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |0|    octetDeltaCount = 1      |       Field Length = 4        |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

Figure 7: IPFIX Metadata template Encoding
An encoded IP fix transport message will be:

   0                   1                   2                   3
   0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |       Version Number          |            Length             |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                           Export Time                         |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                       Sequence Number                         |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                    Observation Domain ID                      |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |          Set ID = 256         |          Length = 64          |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                          192.0.2.12                           |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                          192.0.2.254                          |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                          192.0.2.1                            |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                             5009                              |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                            5344385                            |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

Figure 8: IPFIX Metadata Encoding
3. Metadata transport service
These different use cases for Metadata generate some requirements on the reliability capabilities of the Metadata transport service they will rely on.
De facto Service Function Chain proposals such as NSH and SCH define a mechanism for sharing information along the Chains. It is thus important to define this transport service in term of reliability.
The primary focus of these proposal are the Metadata elements playing a role in the Service Function Chain deployment to apply policies on incoming traffic at some relevant Service Functions.
For example, Network Orchestration is expected to be a significant driver for deployment of Network Services. It relies on Service Level abstractions such as Group Policies, Contracts and Services as an input for the orchestration of Service Function Chains. Specific metadata attributes such as L4-L7 fields are used as classification elements or filters to funnel packets into chains.
Service Functions may have other needs for Metadata, for example Event Based Metadata tied to some subscriber related state changes. The complexity and length of these elements cannot be forseen as limited, neither can be how critical it is that they are safely carried throughout a chain. In  the following Section we propose to take avantage of Congruent Metadata Transport can be used, possibly reliably, to address these needs.
3.1. Proposed Metadata transport mechanisms
.
3.1.3. Metadata transport analysis
Both NSH and SCH proposals support both inbound and offline out-of-band metadata transport.

  1.  In-band: the metadata can be included directly as a value in some of the NSH Context Header Fields. It is the preferred transport model for SCH.

In such case, when a particular field is always set to the same value for all packets transported by the chain instance, then the metadata transport service is in effect reliable.

Similarly, all the packet for a particular flow (defined by its 5 tupple), could receive the same metadata value. The metadata transport service is also reliable, provided that the value is understood to be attached to a flow.

The general case however, occurs when the metadata varies from packet to packet in a flow. The value is then tied to a specific packet. Here the transport service is not reliable. A retransmission of a particular packet would not necessarily lead it to carry the same metadata value.
  2.  Out-of-band: a reference for some metadata is sent along a packet so that it can be used via an interaction with a controller entity. It is the preferred model for NSH, but could also be used by SCH.

As for the in-band case, the metadata referred to indirectly can be transmitted reliably, when its value remains the same for a chain or a flow in a chain. a chain or a flow in a chain.

If however, the correlation Id passed changes over time, or if the Metadata itself cahanges then the correct Metadata may not be efficiently retrieved by some Service Functions.

We can see that NSH and SCH do not address the need for a reliable transport service for metadata, independantly of the reliability characteristic of the transport service used for the data packets.
Conventions can be used as particular cases when some metadata pertains to a specific chain or a flow in the chain, and when its value does not change overtime.
Such conventions however are weak. They would suppose that some mechanism exists to ensure/monitor that they are followed. And some exceptions mechanism would be required to deal with error cases.
3.2. Support for Congruent out-of-band transport service
Congruent out-of-band metadata sharing can be required for some types of Metadata exchanges. It has the advantage of clearly tying the metadata to the chain and not to a specific packet, and to avoid payload fragmentation issues.
Up to draft 2, NSH did not allow for long inline metadata transport. Four 32 bits context fields are reserved for that purpose, and seem best suited for offline Metadata sharing, or to transport predefined policy identifiers.
NSH (since draf 3) as well as SCH could allow for metadata transport, either tied to a packet or possibly tied to the chain, when used without payload, as signaling messages.
SCH however stipulates that in case the Path Identifier and SF Index fields shall be set to zero for transmit and ignored upon receipt, when the SCH packet will contain only metadata. So congruent out-of-band metadata, transporting Metadata hop to hop to the various Service Function in the chain, does not seem to be supported.
NSH supports inline variable size metadata. It does not mentions explicitly that congruent out-of-band metadata can be used.
3.3. Reliable transport service
Some metadata will need a reliable transport service to be shared inline, as well as offline.
A protocol SCTP provides such a service and has the interesting characteristic to be packet based, as opposed to stream based like TCP.
SCTP carries a sequence number and support retransmission and congestion control.
Figure 12 shows how SCTP is used hop by hop between SFF in a chain to transport Metadata reliably.

                     |1  -----   |n        |21  ---- |2m
                   +---+---+   +---+---+   +-+---+   +--+-----+
                   | SF#1  |   |SF#n   |   |SF#i1|   |SF#im   |
                   |       |   |       |   |     |   |        |
                   +---+---+   +---+---+   +--+--+   +--+--+--+
                       :           :          :         :  :
                       :           :          :         :  :
                        \         /            \       /
    +-----------+ SCTP   +--------+    SCTP     +---------+
-- >| Chain     | <--->  | SFF    |    <--->    | SFF     | ---->
    |classifier |        |Node-1  |             | Node-i  |
    +-----------+        +----+---+             +----+--+-+
                  \           |                     /
                   \          | SFC Encapsulation  /
                    \         |                   /
                  ,........................................
                 /                                         \
                /                                           \
               |                      Network                |
                \                                           /
                 \........................................./

Figure 12: SCTP for Reliable Metadata transport
SCTP SHOULD be used in the context of Congruent out of band for reliable metadata sharing.
For reliable Metadata transport, the SFF along the chain MUST route the metadata received over SCTP to the next SF in the chain. For this SCTP MUST be encapsulated in the SFC header.
    The SFF MUST make the received metadata available to its SF.ti

---

From: "Joel M. Halpern" <[email-masked]>
Date: Thu, 10 Jul 2014 14:22:43 -0400

I look forward to seeing your draft.

One of the reasons I like including the metadata with the underlying 
packet is that they then share fate.  If the packet gets lost, the 
metadata disappears with it, which is frequently fine.  And if the 
packet is probably delviered down the service path, then so is the metadata.

Yours,
Joel

On 7/10/14, 2:15 PM, Nicolas BOUTHORS wrote:
> B Rijsman has done a great job showing various option for Metadata
> sharing. Still there are 2 aspects that I would like to address further
>
> *1. Problem Space*
> * Metadata representation*
> Metadata can be extremely varied in term of usage and content. It can
> represent the result of Deep Packet Inspection (DPI) performed on the
> traffic for example by the Classifier Service Function at the ingress of
> the Service Function Chain. It can contain information collected about
> the end user such as a policy identifier, a category or even represent
> an event related to the end-user session.
> This information can be used by Service Functions on the chain, as well
> as by monitoring entities responsible to track usage for example in the
> Network Function Virtualization environment to feed a VNF manager or an
> Orchestrator.
> Metadata being information shared by many network entities needs some
> means to be represented it in all of its dimensions: type definition,
> encoding and scope.
> * Metadata transport service*
> As expected from service chain proposals, both NSH and SCH proposals
> define some means to carry metadata between Service Functions in a
> Chain. They can be classified as follow:
> Dataplane Metadata:
> are defined in the Service Function Chaining Problem Statement document.
> They are not considered to be part of the forwarding information of the
> SFC header. However they are expected to carry the result of antecedent
> classification, allowing Service Functions to take local policy
> decisions based on their values.
>
> As such, they could also be interpreted directly by Service Function
> Forwarder to steer traffic to various Service Functions.
> Offline Metadata:
> Beyond Dataplane Metadata, Offline Metadata can be shared between
> Service Functions in a chain, using out of band, congruent or not, or
> hybrid models described in [RIJSMAN]
>
> The hybrid model for example, defined in [RIJSMAN], utilizes the SFC
> header to transport some key values for correlation purposes. These
> Correlation Ids can be used by the Service Functions to recover the full
> associated contextual information.
> Metadata, directly or indirectly, are transported hop by hop along a
> chain, in association to end-user traffic, the data payload of the SFC
> packets.
> How these metadata are transported over a chain matters. Passing
> metadata directly or indirectly along packets is a service that must be
> analyzed from a reliability point of view.
>
> Reliability requirements may vary depending on the nature of the
> metadata transported. Past experience for example in Mobile Network and
> data center with AAA Radius have shown that contextual information
> replication to various service function was indeed sensitive to packet
> loss events, and that adhoc solutions had to be implemented to detect them.
> End to end TCP retransmission of packet lost does not ensure that
> associated Metadata will be reinserted in retransmitted packet. In
> addition, Event triggered metadata may have to be sent immediately on a
> chain even though no end-user traffic is being transmitted
> Not all metadata needs a reliable transport. Repeated inline metadata
> can be used to cover several use cases, and some metadata loss can be
> acceptable.
> Still, a reliable transport service for Metadata in SFC is expected. To
> this effect, an implementation is suggested in Section 3.3
> *2. Metadata representation*
> Metadata definition is that it provides contextual information about the
> data packets which traverse a Service Function Chain. This must be
> understood broadly.
> Metadata can contain the result of traffic classification by Deep Packet
> Inspection (DPI). For example as an Application Id information which is
> tied to a traffic flow. There could be multiple flows with different
> application ids, in a chain.
> Metadata can also contain the result of DPI data extraction, such as
> identify requested URL in HTTP. Such information can be passed to
> certain SF down the chain such as a URL filtering function.
> Metadata can contain some punctual event information collected at the
> Ingress point of the chain and expected to be passed to all elements in
> the chain. Here this information may be triggered externally and
> generated only once, and be related to the tenant or the subscriber.
> Metadata representation involves the definition of a set of information
> elements types and the encoding rules for their values.
> Metadata representation can sometimes be performed by a single
> individual field with associated type and format. However, it is not
> always enough.
>
>   * Some addition fields may be required to describe the scope of the
>     metadata itself. This can be any information element allowing to
>     define the context of the associated metadata value. For example a
>     throughput metadata field can have a port number and a switch
>     address as its Scope information.
>
> We propose IPFIX as a preferred means to represent Metadata in Service
> Chain messages for Out-of-band, Congruent or not; Metadata sharing.
> *2.1. Metadata Representation Requirements*
> Mandatatory Dataplane Metadata is always part of the SFC header, it is
> thus reasonable to consider that its representation scheme will be
> implicit: based on what the SFC protocol will dictate, their position in
> the SFC header is sufficient for the receiving end to infer their type
> and encoding scheme. For example, Context Header Fields in NSH are 32
> bit fields.
> However, it will not be the case for all metadata transported. Offline
> metadata, including congruent out-of-band metadata still need to be
> represented explicitly. This section addresses their specific case.
> *2.1.1. Metadata Encoding requirements*
> These requirements are applicable to out-of-band metadata (Congruent or
> not). It could be applicable with SCH on optional in-line metadata fields.
> For interoperability purposes, metadata encoding MUST allow the
> receiving entity to identify the type and value of the information
> received as metadata
> Metadata encoding MUST allow for encoding techniques supporting well
> known types and fields as well as proprietary extensions.
> A receiving entity MUST be able to identify when incoming metadata type
> is unknown and MUST have a defined default action to handle it.
> A piece of information may need multiple fields to be described. For
> example a tenant id and an ip address can be used to identify an server
> in a data center uniquely.
> These groups of information have to be exchanged collectively, in a
> single message. In this case, a sending entity MUST specify that it is
> sending a set of metadata in a message.
> This set of transported metadata elements MUST be specified under the
> form of a metadata template document uniquely defined for the chain.
> A receiving entity MUST be able to detect if an incoming message
> contains its expected set of metadata elements.
> *2.1.2. Metadata Scope requirement*
> A piece of information may have to be qualified by some attributes
> allowing to identify its particular scope. For example in a Gi
> environment, the radio type in use may be used as a scoping criteria for
> a jitter or latency measurement in a video traffic transported in a
> particular service chaing.
> Scope can apply to some individual metadata elements or to a set of
> metadata elements. How a scope applies to a set of transported metadata
> elements should be defined by a specification under the form of a
> metadata template document uniquely identified for the chain.
> *2.2. IPFIX Metadata representation*
> So far, unordered sequences of Type Length Value encoded fields have
> been proposed to transport metadata. It is not clear how structured
> types are supported, and no distinction is done between the metadata
> values and their scoping values. Although the SCH proposal provides an
> optional 24-bit Organizational Unique Identifier, there is no namespace
> mechanism allowing to separate type definition spaces per Tenants or per
> chain.
> A natural candidate to leverage is IPFIX [RFC7011]: IPFIX is a means for
> transmitting Traffic Flow information over the network. In order to
> transmit Traffic Flow information, it provides a common representation
> of flow data and a standard means of communicating them.
> Metadata collected by Network Node and Service Node SHOULD be encoded in
> template following the principles described in IPFIX[RFC7011].
> An IPFIX Message consisting of interleaved Template, Data, and Options
> Template Sets, as shown in Figure 1. Here, Template and Options Template
> Sets , which are optional, are shown.
>
>
> The Template Set describes the data transmitted in the following Data
> Set. It is an optional component of the message. The value of the
> metadata is encoded in the first Data Set. This Data Set contains a
> template Id field as a reference to its defining Template Set.
> The Options Template Set describes the data to be transmitted as scope
> information. It is an optional component of the message. The value of
> the scope information is encoded in the second Data Set element. It no
> scope information is present, then only the first Data Set is present in
> the message.
> The Option Template Set and following Data Set are used to describe the
> scope of the metadata transmitted. For example, the metadata collected
> is relevant to a PDP Context or a particular line card of a particular
> switch.
> *
> *
> *2.2.4.3. Example:*
> The Metadata Exporting Process creates a Template Record with a few
> Information Elements: amongst other things, the Application Id. For example:
>
> Scope information is useful when sending metadata offline, as it can
> contain information related to the chain and possibly to the flow for
> which this metadata record is relevant. Here servicePath and
> serviceIndex are thus included in the Template.
> *2.2.4.4. IPFIX encoding and template provisioning:*
> Note that an XML representation of IPFIX template record could be
> defined and used to provision Service Functions.
>
>   0                   1                   2                   3
> *3. Metadata transport service*
> These different use cases for Metadata generate some requirements on the
> reliability capabilities of the Metadata transport service they will
> rely on.
> De facto Service Function Chain proposals such as NSH and SCH define a
> mechanism for sharing information along the Chains. It is thus important
> to define this transport service in term of reliability.
> The primary focus of these proposal are the Metadata elements playing a
> role in the Service Function Chain deployment to apply policies on
> incoming traffic at some relevant Service Functions.
> For example, Network Orchestration is expected to be a significant
> driver for deployment of Network Services. It relies on Service Level
> abstractions such as Group Policies, Contracts and Services as an input
> for the orchestration of Service Function Chains. Specific metadata
> attributes such as L4-L7 fields are used as classification elements or
> filters to funnel packets into chains.
> Service Functions may have other needs for Metadata, for example Event
> Based Metadata tied to some subscriber related state changes. The
> complexity and length of these elements cannot be forseen as limited,
> neither can be how critical it is that they are safely carried
> throughout a chain. In  the following Section we propose to take
> avantage of Congruent Metadata Transport can be used, possibly reliably,
> to address these needs.
> *3.1. Proposed Metadata transport mechanisms*
> .
> *3.1.3. Metadata transport analysis*
>  1. In-band: the metadata can be included directly as a value in some of
>     the NSH Context Header Fields. It is the preferred transport model
>     for SCH.
>
>     In such case, when a particular field is always set to the same
>     value for all packets transported by the chain instance, then the
>     metadata transport service is in effect reliable.
>
>     Similarly, all the packet for a particular flow (defined by its 5
>     tupple), could receive the same metadata value. The metadata
>     transport service is also reliable, provided that the value is
>     understood to be attached to a flow.
>
>     The general case however, occurs when the metadata varies from
>     packet to packet in a flow. The value is then tied to a specific
>     packet. Here the transport service is not reliable. A retransmission
>     of a particular packet would not necessarily lead it to carry the
>     same metadata value.
>  2. Out-of-band: a reference for some metadata is sent along a packet so
>     that it can be used via an interaction with a controller entity. It
>     is the preferred model for NSH, but could also be used by SCH.
>
>     As for the in-band case, the metadata referred to indirectly can be
>     transmitted reliably, when its value remains the same for a chain or
>     a flow in a chain. a chain or a flow in a chain.
>
>     If however, the correlation Id passed changes over time, or if the
>     Metadata itself cahanges then the correct Metadata may not be
>     efficiently retrieved by some Service Functions.
>
> We can see that NSH and SCH do not address the need for a reliable
> transport service for metadata, independantly of the reliability
> characteristic of the transport service used for the data packets.
> Conventions can be used as particular cases when some metadata pertains
> to a specific chain or a flow in the chain, and when its value does not
> change overtime.
> Such conventions however are weak. They would suppose that some
> mechanism exists to ensure/monitor that they are followed. And some
> exceptions mechanism would be required to deal with error cases.
> *3.2. Support for Congruent out-of-band transport service*
> Congruent out-of-band metadata sharing can be required for some types of
> Metadata exchanges. It has the advantage of clearly tying the metadata
> to the chain and not to a specific packet, and to avoid payload
> fragmentation issues.
> Up to draft 2, NSH did not allow for long inline metadata transport.
> Four 32 bits context fields are reserved for that purpose, and seem best
> suited for offline Metadata sharing, or to transport predefined policy
> identifiers.
> NSH (since draf 3) as well as SCH could allow for metadata transport,
> either tied to a packet or possibly tied to the chain, when used without
> payload, as signaling messages.
> SCH however stipulates that in case the Path Identifier and SF Index
> fields shall be set to zero for transmit and ignored upon receipt, when
> the SCH packet will contain only metadata. So congruent out-of-band
> metadata, transporting Metadata hop to hop to the various Service
> Function in the chain, does not seem to be supported.
> NSH supports inline variable size metadata. It does not mentions
> explicitly that congruent out-of-band metadata can be used.
> *3.3. Reliable transport service*
> A protocol SCTP provides such a service and has the interesting
> characteristic to be packet based, as opposed to stream based like TCP.
> For reliable Metadata transport, the SFF along the chain MUST route the
> metadata received over SCTP to the next SF in the chain. For this SCTP
> MUST be encapsulated in the SFC header.
> _______________________________________________
> sfc mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/sfc
>

---

From: Changcheng Huang <[email-masked]>
Date: Thu, 10 Jul 2014 11:53:26 -0700

I am concerned with the overhead metadata may create. Internet packet 
sizes tend to be bimodal where more than 50% of packets have very small 
packet sizes. Carrying large amounts of overhead will reduce throughput 
significantly. Making metadata optional may be a good choice.

Regards,

Chang

---

From: "Joel M. Halpern" <[email-masked]>
Date: Thu, 10 Jul 2014 15:00:46 -0400

I would hope that we will not be adding large amounts of metadata to 
packets.  There are many reasons why that may cause complications.
That is not the same as not carrying some metadata with packets.

Yours,
Joel

---

From: Sam Aldrin <[email-masked]>
Date: Thu, 10 Jul 2014 12:02:23 -0700

> significantly. Making metadata optional may be a good choice.
Doesn't that depend only if the SFC spans across the internet?
Otherwise it (overhead) should be limited to SFC's domain, which may not
span across internet, no?

cheers
-sam

>> packet is that they then share fate.  If the packet gets lost, the metadata
>> disappears with it, which is frequently fine.  And if the packet is
>> probably delviered down the service path, then so is the metadata.
>>
>> Yours,
>> Joel
>>

---

From: Changcheng Huang <[email-masked]>
Date: Thu, 10 Jul 2014 12:47:15 -0700

Even the SFC is limited to a specific domain (such as a datacenter), the 
packet sizes still follow the same distribution. The bandwidth within a 
datacenter is still a bottleneck part due to traffic patterns such as 
incast.

Chang

On 07/10/2014 12:02 PM, Sam Aldrin wrote:
>
>
> On Thu, Jul 10, 2014 at 11:53 AM, Changcheng Huang 
> <[email-masked] <mailto:[email-masked]>> wrote:
>
>     I am concerned with the overhead metadata may create. Internet
>     packet sizes tend to be bimodal where more than 50% of packets
>     have very small packet sizes. Carrying large amounts of overhead
>     will reduce throughput significantly. Making metadata optional may
>     be a good choice.
>
> Otherwise it (overhead) should be limited to SFC's domain, which may 
> not span across internet, no?
>
> cheers
> -sam
>
>
>     Regards,
>
>     Chang
>
>
>         One of the reasons I like including the metadata with the
>         underlying packet is that they then share fate.  If the packet
>         gets lost, the metadata disappears with it, which is
>         frequently fine.  And if the packet is probably delviered down
>         the service path, then so is the metadata.
>
>         Yours,
>         Joel
>
>             Metadata transport is a feature provided by SFC that needs
>             some
>             additional thoughts as well.
>
>             We will probably post a draft on the subject, but this can
>             be done after
>             Toronto has started. So for those
>                 |        | |          | |         |     | Set       |
>             |         | |
>             The Options Data Record associated with the examples above
>             would contain
>             the Scoping inforamtion:
>
>                   Scope:
>                           - servicePath,
>                           - serviceIndex,
>                           - applicationId,
>                           - applicationName
>                 |                           Export Time              
>                 |
>                 |                       Sequence Number              
>                 |
>                 |                    Observation Domain ID            
>                  |
>             Both NSH and SCH proposals support both inbound and
>             offline out-of-band
>             metadata transport.
>
>             Some metadata will need a reliable transport service to be
>             shared
>             inline, as well as offline.
>             SCTP carries a sequence number and support retransmission
>             and congestion
>             control.
>             Figure 12 shows how SCTP is used hop by hop between SFF in
>             a chain to
>             transport Metadata reliably.
>
>                                   |1  -----   |n        |21  ---- |2m
>             SCTP SHOULD be used in the context of Congruent out of
>             band for reliable
>             metadata sharing.
>             [email-masked] <mailto:[email-masked]>
>         [email-masked] <mailto:[email-masked]>
>     [email-masked] <mailto:[email-masked]>

---

From: Sam Aldrin <[email-masked]>
Date: Thu, 10 Jul 2014 13:22:56 -0700

I agree . As SFC is established in a controlled domain, mitigating options are better than general internet. Again need to see how much of an overhead it really causes and it's side effects.

Sam

Sent from my iPad

> On Jul 10, 2014, at 12:47 PM, Changcheng Huang <[email-masked]> wrote:
> 
> Even the SFC is limited to a specific domain (such as a datacenter), the packet sizes still follow the same distribution. The bandwidth within a datacenter is still a bottleneck part due to traffic patterns such as incast.
> 
> Chang
> 
>>> On Thu, Jul 10, 2014 at 11:53 AM, Changcheng Huang <[email-masked]> wrote:
>>> I am concerned with the overhead metadata may create. Internet packet sizes tend to be bimodal where more than 50% of packets have very small packet sizes. Carrying large amounts of overhead will reduce throughput significantly. Making metadata optional may be a good choice.
>> Otherwise it (overhead) should be limited to SFC's domain, which may not span across internet, no?
>> 
>> cheers
>> -sam
>>> 
>>> Regards,
>>> 
>>> Chang
>>> 
>>> 
>>>> One of the reasons I like including the metadata with the underlying packet is that they then share fate.  If the packet gets lost, the metadata disappears with it, which is frequently fine.  And if the packet is probably delviered down the service path, then so is the metadata.
>>>> 
>>>> Yours,
>>>> Joel
>>>>

---

From: Changcheng Huang <[email-masked]>
Date: Thu, 10 Jul 2014 13:46:02 -0700

The most common small packets are TCP Ack packets which are around 64 
bytes long including headers from various layers (MAC, IP, TCP) and some 
padded payloads to make it satisfy minimum packet size requirement. 
Adding metadata to this kind of packets does not make sense to me.

Chang

On 07/10/2014 01:22 PM, Sam Aldrin wrote:
> I agree . As SFC is established in a controlled domain, mitigating 
> options are better than general internet. Again need to see how much 
> of an overhead it really causes and it's side effects.
>
> Sam
>
> Sent from my iPad
>
> On Jul 10, 2014, at 12:47 PM, Changcheng Huang <[email-masked] 
> <mailto:[email-masked]>> wrote:
>
>> Even the SFC is limited to a specific domain (such as a datacenter), 
>> the packet sizes still follow the same distribution. The bandwidth 
>> within a datacenter is still a bottleneck part due to traffic 
>> patterns such as incast.
>>
>> Chang
>>
>>>     will reduce throughput significantly. Making metadata optional
>>>     may be a good choice.
>>>
>>>         underlying packet is that they then share fate.  If the
>>>         packet gets lost, the metadata disappears with it, which is
>>>         frequently fine.  And if the packet is probably delviered
>>>         down the service path, then so is the metadata.
>>>
>>>         Yours,
>>>         Joel
>>>
>>>             Metadata transport is a feature provided by SFC that
>>>             needs some
>>>             We will probably post a draft on the subject, but this
>>>             can be done after
>>>               * Metadata may need multiple fields transported
>>>             together to
>>>                 represented their values.
>>>                 |        | |          | |         | | Set       | |
>>>                     | |
>>>             The Options Data Record associated with the examples
>>>             above would contain
>>>             the Scoping inforamtion:
>>>
>>>                   Scope:
>>>                           - servicePath,
>>>                           - serviceIndex,
>>>                           - applicationId,
>>>                           - applicationName
>>>               0                   1                   2            
>>>                   3
>>>             Some metadata will need a reliable transport service to
>>>             be shared
>>>             SCTP carries a sequence number and support
>>>             retransmission and congestion
>>>             control.
>>>             Figure 12 shows how SCTP is used hop by hop between SFF
>>>             in a chain to
>>>                                 |       |   |       | |     |   |

---

From: "Joel M. Halpern" <[email-masked]>
Date: Thu, 10 Jul 2014 16:50:03 -0400

If the packet does not carry enough information to identify the 
subscriber, as has been described in several use casess, there appears 
to be no choice other than to attach some subscriber identifying 
aidditional information tothe packet.  Sending it separately from the 
packet simply won't work.
That is clearly metadata, and it seems clear to me that it needs to be 
attached to the packet.

That does not mean that there are not other cases where out-of-band 
delivery of various sorts may also be useful.  I am not against using 
that as well.

Yours,
Joel

---

From: "Paul Quinn (paulq)" <[email-masked]>
Date: Fri, 11 Jul 2014 09:07:30 +0000

On Jul 10, 2014, at 4:50 PM, Joel M. Halpern <[email-masked]> wrote:

> If the packet does not carry enough information to identify the subscriber, as has been described in several use casess, there appears to be no choice other than to attach some subscriber identifying aidditional information tothe packet.  Sending it separately from the packet simply won't work.
> That is clearly metadata, and it seems clear to me that it needs to be attached to the packet.
> 
> That does not mean that there are not other cases where out-of-band delivery of various sorts may also be useful.  I am not against using that as well.
> 

Right, carrying metadata per packet and having out of band information are not mutually exclusive.