Subject: [ippm] Fw: New Version Notification for draft-elkins-ippm-pdm-option-03.txt

From: Nalini Elkins <[email-masked]>
Date: Sun, 15 Feb 2015 16:20:25 +0000 (UTC)

Folks,
We have posted a new draft of our PDM proposal.   Thanks to all for your comments in Hawaii.
There were a number of questions posed, which we have categorized and answered.   If thereare others, please let us know.    
The posting got too big, so I have created an attachment.  Hope that will get through.
Thanks again to all.  
Question 1.  PDM does not have enough variables to actually diagnose problems.Question 2.  PDM has too many fields.  Are they all necessary,Question 3.  PDM will only work for simple applications with one send and one receive.Question 4.  Why is the proposal for an IPv6 extension header rather than a TCP option?   Only TCP is important.Question 5.  PDM creates too much overhead.Question 6.  Where in stack and process will timing be done?   Question 7.  Do middle boxes need to process PDM? Thanks,

Nalini Elkins
Inside Products, Inc.
(831) 659-8360
www.insidethestack.com

    ----- Forwarded Message -----
  From: "[email-masked]" <[email-masked]>
 To: Nalini Elkins <[email-masked]>; Rob Hamilton <[email-masked]>; Michael S. Ackermann <[email-masked]>; Nalini Elkins <[email-masked]>; Robert Hamilton <[email-masked]>; Michael Ackermann <[email-masked]> 
 Sent: Sunday, February 15, 2015 8:05 AM
 Subject: New Version Notification for draft-elkins-ippm-pdm-option-03.txt

A new version of I-D, draft-elkins-ippm-pdm-option-03.txt
has been successfully submitted by Nalini Elkins and posted to the
IETF repository.

Name:        draft-elkins-ippm-pdm-option
Revision:    03
Title:        IPPM Considerations for the IPv6 PDM Destination Option
Document date:    2015-02-14
Group:        Individual Submission
Pages:        25
URL:            http://www.ietf.org/internet-drafts/draft-elkins-ippm-pdm-option-03.txt
Status:        https://datatracker.ietf.org/doc/draft-elkins-ippm-pdm-option/
Htmlized:      http://tools.ietf.org/html/draft-elkins-ippm-pdm-option-03
Diff:          http://www.ietf.org/rfcdiff?url2=draft-elkins-ippm-pdm-option-03

Abstract:
  To assess performance problems,  measurements based on optional
  sequence numbers and timing may be embedded in each packet.  Such
  measurements may be interpreted in real-time or after the fact. An
  implementation of the existing IPv6 Destination Options extension
  header, the Performance and Diagnostic Metrics (PDM) Destination
  Options extension header has been proposed in a companion document.
  This document specifies the field limits, calculations, and usage of
  the PDM in measurement.

Please note that it may take a couple of minutes from the time of submission
until the htmlized version and diff are available at tools.ietf.org.

The IETF Secretariat

---

From: "Scheffenegger, Richard" <[email-masked]>
Date: Fri, 27 Feb 2015 09:28:59 +0000

Hi,

I’ve been asked to forward my response to the list, so here we go.

Best regards,
   Richard

From: Scheffenegger, Richard
Sent: Dienstag, 24. Februar 2015 16:24
To: 'Nalini Elkins'
Cc: MORTON C (AL); Ackermann Michael; Rob Hamilton
Subject: RE: New Version Notification for draft-elkins-ippm-pdm-option-03.txt

Hi Nalini,

thanks a lot; i concur with many of these statements (in fact, I’m working on a proposal for TCP to have in idependent per-segment tag, not timestamp, to be useful for better tcp loss recovery – especially lost retransmissions, but also helping in case of reordering and duplication. So I do see why you’d like many of that data to be readily available for analysis).

I can remember that one of my questions was aimed at multicast / any-any flows; from past experience, I believe that some medical equipment has data flows in such any-any manners – using either IP subnet broadcast addresses, or proper IP multicast addresses. Do you propose these to be a collection of many uni-directional flows, or try to have PDM making sense of the 4-tuple and track the responses accordingly? I think that should be made clearer in the draft.

As to the answer to Q5: My point was not so much the wire-overhead, but the end host overhead when it comes to hardware-assisted processing of
IPv6 extension headers; This also comes into play with the answer to Q6 – naturally, the PDM header (and timing) should be done as close (in time) as possible to the packet actually hitting the wire. However, if current TCP segmentation offload is any hint, a host may end up zero-copying a bunch (100s of kB data) to the NIC, which then segments that all, and prepares the IP/ TCP headers using fixed templates (only the few fields that need adjusting for TCP are actually adjusted; everything else, like IP options or TCP option (timestamp) will remain the same for one such transfer, which may stretch dozends of packets…

TSO is quite important for end hosts to achive wire-speed; touching each IP header (PDM extension header) individually sounds like to have performance implication, as long as there is no native hardware support for it…

Best regards,
  Richard

From: Nalini Elkins [mailto:[email-masked]]
Sent: Montag, 16. Februar 2015 14:04
To: Scheffenegger, Richard
Cc: MORTON C (AL); Ackermann Michael; Rob Hamilton
Subject: Fw: New Version Notification for draft-elkins-ippm-pdm-option-03.txt

Richard,

I am sending you the latest version of our PDM draft as well as an attachment with answers to questions that you and others have asked.   Thank you so much for your comments.

Please let me know if you have any more issues for us.

Question 1.  PDM does not have enough variables to actually diagnose problems.
Question 2.  PDM has too many fields.  Are they all necessary,
Question 3.  PDM will only work for simple applications with one send and one receive.
Question 4.  Why is the proposal for an IPv6 extension header rather than a TCP
             option?   Only TCP is important.
Question 5.  PDM creates too much overhead.
Question 6.  Where in stack and process will timing be done?
Question 7.  Do middle boxes need to process PDM?

Question 1. PDM does not have enough variables to actually diagnose problems.

The purpose of the PDM is not to supplant all the variables present
in all other headers but to provide data which is not available or
very difficult to get.   The way PDM would be used is by a technician
(or tool) looking at a packet capture.   Within the packet capture,
they would have available to them the layer 2 header, IP header (v6
or v4), TCP, UCP, ICMP, SCTP or other headers.   All information
would be looked at together to make sense of the packet flow.   The
technician or processing tool (for example, Wireshark)
could analyze, report or ignore the data from PDM, as necessary.

For an example of how PDM can help with TCP retransmit problems,
please look at the answers to Question 3.

Question 2.  PDM has too many fields.  Are they all necessary,

Yes.   Please take a look at the answers to Question 3.

Question 3.  PDM will only work for simple applications with one send and one receive.

This is not true.  Please find below flow analysis for one-way flow, multiple
send flow, and multiple send flow with errors.

PDM Flow - One Way Traffic

In the case of a one-way flow, one might see the following:

   Packet   Sender      PSN            PSN            Delta      Delta
                     This Packet    Last Recvd     Last Recvd  Last Sent
   =====================================================================
   1        Server       1              0              0            0
   2        Server       2              0              0            5
   3        Server       3              0              0           12
   4        Server       4              0              0           20

   What does this mean and how is it useful?

   In a one-way flow, only the Delta Last Sent will be seen as used.
   Recall, Delta Last Sent is the difference between the send of one
   packet from a device and the next.  This is a measure of throughput
   for the sender - according to the sender's point of view.  That is,
   it is a measure of how fast is the application itself (with stack
   time included) able to send packets.

   How might this be useful?  If one is having a performance issue at
   the client and sees that packet 2, for example, is sent after 5
   microseconds from the server but takes 3 minutes to arrive at the
   destination,  then one may safely conclude that there are delays in
   the path other than at the server which may be causing the delivery
   issue of that packet.  Such delays may include the network links,
   middle-boxes, etc.

   Now, true one-way traffic is quite rare.   What people often mean by
   "one-way" traffic is an application such as FTP where a group of
   packets (for example, a TCP window size worth) is sent, then the
   sender waits for acknowledgment.  This type of flow would actually
   fall into the "multiple-send" traffic model.

PDM Flow - Multiple Send Traffic

   Assume that two packets are sent with each send from the server.

   Packet   Sender      PSN            PSN            Delta      Delta
                     This Packet    Last Recvd     Last Recvd  Last Sent
   =====================================================================
   1        Server       1              0              0           0
   2        Server       2              0              0           5
   3        Client       1              1             20           0
   4        Server       3              1             10          15

   How might this be used?

   Notice that in packet 3, the client has a value of Delta Last
   received of 20.   Recall that Delta Last Received is the Send time of
   packet 3 - receive time of packet 2.   So, what does one know now?
   In this case, Delta Last Received is the processing time for the
   Client to send the next packet.

   How to interpret this depends on what is actually being sent.
   Remember, PDM is not being used in isolation, but to supplement the
   fields found in other headers.   Let's take some examples:

   1.  Client is sending a standalone TCP ACK.   One would find this by
   looking at the payload length in the IPv6 header and the TCP
   Acknowledgement field in the TCP header.   So, in this case, the
   client is taking 20 units to send back the ACK.   This may or may not
   be interesting.

   2.  Client is sending data with the packet.  Again, one would find
   this by looking at the payload length in the IPv6 header and the TCP
   Acknowledgement field in the TCP header.   So, in this case, the
   client is taking 20 units to send back data.   This may represent
   "User Think Time".   Again, this may or may not be interesting, in
   isolation.   But, if there is a performance problem receiving data at
   the server, then taken in conjunction with RTT or other packet timing
   information, this information may be quite interesting.

   Of course, one also needs to look at the PSN Last Received field to
   make sure of the interpretation of this data.   That is,  to make
   sure that the Delta Last Received corresponds to the packet of
   interest.

   The benefits of PDM are that we have such information available in a
   uniform manner for all applications and all protocols without
   extensive changes required to applications.

   PDM Flow - Multiple Send with Errors

   One might wonder if all of the functions of PDM might be better
   suited to TCP or a TCP option.   Let us take the case of how PDM may
   help in a case of TCP retransmissions in a way that TCP options or
   TCP ACK / SEQ would not.

   Assume that three packets are sent with each send from the server.

   From the server, this is what is seen.

   Pkt Sender    PSN        PSN       Delta      Delta    TCP     Data
               This Pkt  LastRecvd  LastRecvd  LastSent   SEQ    Bytes
   =====================================================================
   1   Server      1        0           0          0      123     100
   2   Server      2        0           0          5      223     100
   3   Server      3        0           0          5      333     100

   The client however, does not get all the packets.  From the client,
   this is what is seen for the packets sent from the server.

   Pkt Sender    PSN        PSN       Delta      Delta   TCP    Data
              This Pkt  LastRecvd  LastRecvd  LastSent   SEQ    Bytes
   =====================================================================
   1   Server     1         0          0          0        123   100
   2   Server     3         0          0          5        333   100

   Let's assume that the server now retransmits the packet.
   (Obviously, a duplicate acknowledgment sequence for fast retransmit
   or a retransmit timeout would occur.  To illustrate the point, these
   packets are being left out.)

   So, then if a TCP retransmission is done, then from the client, this
   is what is seen for the packets sent from the server.

   Pkt Sender    PSN        PSN       Delta      Delta    TCP    Data
              This Pkt  LastRecvd  LastRecvd   LastSent   SEQ    Bytes
   =====================================================================
   1   Server    4          0           0          30    223      100

   The server has resent the old packet 2 with TCP sequence number of
   223.   The retransmitted packet now has a PSN This Packet value of 4.
   The Delta Last Sent is 30.   That is the time between sending the
   packet with PSN of 3 and this current packet.

   Let's say that packet 4 STILL does not make it.  Then, after some
   amount of time (RTO) then the packet with TCP sequence number of 223
   is resent.

   From the client, this is what is seen for the packets sent from the
   server.

   Pkt Sender    PSN        PSN       Delta    Delta     TCP    Data
              This Pkt  LastRecvd  LastRecvd  LastSent   SEQ    Bytes
   =====================================================================
   1   Server    5          0          0        60          223     100

   If now, this packet makes it, one has a very good idea that packets
   exist which are being sent from the server as retransmissions and not
   making it to the client.   This is because the PSN of the resent
   packet from the server is 5 rather than 4.  If we had used TCP
   sequence number alone, we would never have seen this situation.
   Because the TCP sequence number in all situations is 223.

   This situation would be experienced by the user of the application
   (the human being actually sitting somewhere) as a "hangs" or long
   delay between packets.  On large networks, to diagnose problems such
   as these where packets are lost somewhere on the network,  one has to
   take multiple traces to find out exactly where.

   The first thing is to start with doing a trace at the client and the
   server.  So, we can see if the server sent a particular packet and
   the client received it.  If the client did not receive it, then we
   start tracking back to trace points at the router right after the
   server and the router right before the client.  Did they get these
   packets which the server has sent?   This is a time consuming
   activity.

   With PDM,  we can speed up the diagnostic time because we may be able
   to use only the trace taken at the client to see what the server is
   sending.

Question 4.  Why is the proposal for an IPv6 extension header rather than
a TCP option?   Only TCP is important.

Large enterprises have traffic which is non-TCP which will benefit from PDM.

Non-TCP traffic includes:

    -   IBM's Enterprise Extender, which allows its SNA Peer-to-Peer
        Networking (APPN) traffic flow over UDP links
    -   Some WWW traffic flows as UDP packets
    -   TFTP, SNMP, ARP/RARP, OSPF, etc.

TCP applications will also benefit from PDM.   Please look at section xxx.

Question 5.  PDM creates too much overhead.

The total PDM size is only 16 bytes.  16 (optional) bytes should NOT be
an issue on today’s high speed networks.

Below is a table outlining the potential overhead in terms of
   additional time to deliver the response to the end user.

   Packet
   Bytes         RTT           Bytes      Bytes      New   Overhead
   in Packet                 Per Milli    in PDM     RTT
   =====================================================================
   1000       1000 milli         1       16       1016.000  16.000 milli
   1000        100 milli        10       16        101.600   1.600 milli
   1000         10 milli       100       16         10.160    .160 milli
   1000          1 milli      1000       16          1.016    .016 milli

   Below are some examples of actual RTTs for packets traversing large
   enterprise networks.   The first example is for packets going to
   multiple business partners.

   Packet
   Bytes         RTT           Bytes      Bytes      New   Overhead
   in Packet                 Per Milli    in PDM     RTT
   =====================================================================
   1000       17 milli         58         16       17.360   .360 milli

   The second example is for packets at a large enterprise customer
   within a data center.  Notice that the scale is now in microseconds
   rather than milliseconds.

   Packet
   Bytes         RTT           Bytes      Bytes      New   Overhead
   in Packet                 Per Micro    in PDM     RTT
   =====================================================================
    1000       20 micro        50         16       20.320   320 micro

Question 6.  Where in stack and process will timing be done?

The goal is to estimate the time packet data spent on the wire vs. other processing.
Any processing related to packet assembly, header construction, as well as application
time is additional processing which will be unrelated to the measurement of that travel
time, so such processing should be done before the timestamp is taken so as to minimize
any local overhead.
Similarly, when a packet is received, that arrival timestamp is vital in determining packet
travel time, so it is important to minimize local overhead by obtaining that timestamp immediately.

Question 7.  Do middle boxes need to process PDM?

No, and in fact, it is important for accuracy in timing that the middle boxes
DO NOT process or generate PDM data.  Additional intermediary processing would lengthen
the packet delivery time at every hop along the way.
IF an intermediary node over which these packets travel wishes to determine whether
there is a delay between itself and another node, it could open its own connection to
verify that, and then, PDM would be useful on THAT connection.

Thanks,

Nalini Elkins
Inside Products, Inc.
(831) 659-8360
www.insidethestack.com<http://www.insidethestack.com>

----- Forwarded Message -----
From: Nalini Elkins <[email-masked]<mailto:[email-masked]>>
To: IPPM IETF <[email-masked]<mailto:[email-masked]>>
Sent: Sunday, February 15, 2015 8:20 AM
Subject: Fw: New Version Notification for draft-elkins-ippm-pdm-option-03.txt

Folks,

There were a number of questions posed, which we have categorized and answered.   If there
are others, please let us know.

Thanks again to all.

Question 1.  PDM does not have enough variables to actually diagnose problems.
Question 2.  PDM has too many fields.  Are they all necessary,
Question 3.  PDM will only work for simple applications with one send and one receive.
Question 4.  Why is the proposal for an IPv6 extension header rather than a TCP option?   Only TCP is important.
Question 5.  PDM creates too much overhead.
Question 6.  Where in stack and process will timing be done?
Question 7.  Do middle boxes need to process PDM?

Thanks,

Nalini Elkins
Inside Products, Inc.
(831) 659-8360
www.insidethestack.com<http://www.insidethestack.com>

----- Forwarded Message -----
From: "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>
To: Nalini Elkins <[email-masked]<mailto:[email-masked]>>; Rob Hamilton <[email-masked]<mailto:[email-masked]>>; Michael S. Ackermann <[email-masked]<mailto:[email-masked]>>; Nalini Elkins <[email-masked]<mailto:[email-masked]>>; Robert Hamilton <[email-masked]<mailto:[email-masked]>>; Michael Ackermann <[email-masked]<mailto:[email-masked]>>
Sent: Sunday, February 15, 2015 8:05 AM
Subject: New Version Notification for draft-elkins-ippm-pdm-option-03.txt

Abstract:

The IETF Secretariat