Subject: [ippm] Re: OWAMP protocol

From: stanislav shalunov <[email-masked]>
Date: 04 Feb 2005 11:51:09 -0500

Roland,

Your message seems to be mostly about the OWAMP protocol itself rather
than the particular implementation.  I have added [email-masked] (IETF
IP Performance Metrics working group), the IETF WG responsible for the
protocol, draft-ietf-ippm-owdp-17.txt (now in ``publication
requested'' state) to the CC list.

Roland Karch <[email-masked]> writes:

> We have slightly more detailed information about the timesource
> involved in the tests.  Besides the estimated error and the
> timesource (stratum 1?) we also stored the type of the sender's
> timestamps (us vs. ns) and some integrity checks (e.g. packet size).

The protocol needs to be applicable to a wide range of scenarios.
Some system-dependent information about the time source, would,
therefore, be lost in any case.  (Can't include everything for every
possible system.)  I believe that what the protocol has now strikes a
reasonable balance: most implementations should be able to come up
with the information, and it describes most of the important stuff
about the time source.  Note that NTP is not the only possible time
source for OWAMP implementations.

> On second thought, I think it's possible that we just dispose of the
> additional data and make do with just the clock info the protocol
> provides. I have one problem with your choice of format for the
> timestaps though: We'll run into problems in about 30 years from
> now. But that's maybe something that the NTP folks should fix. ;-)

The NTP folks seem to have already done so, or so David Mills says.
The arithmetic with timestamps 30 years from now would need to be a
little funny (add epoch duration if negative), but this is the same
Z/nZ arithmetic that's used in, say, TCP sequence numbers wraparound.
Since the epoch is large enough, there's no need to worry about doing
the right modulo just yet, but that can always be done.

> Excellent, that's (almost) how I would describe a part of our packet
> grouping problem. One thing I'd avoid is to have 0 waiting time between
> test packets. I've seen some interesting effects, just because the
> sender actually queued packets faster than it could send them out on our
> system.

In the protocol specification, this is left up to the user.  The user
can specify zero time between packets (e.g., for packet train
measurements).  If that works poorly, the user is free to specify some
other time.

> That's an interesting way to solve these problems. As I mentioned once,
> we try to avoid collisions of packets completely by either negotiating
> or centrally assigning a global schedule for the measurement slots. I
> can see a way to use OWAMP-control for the latter, centralized case. How
> to efficiently negotiate a test without that entity still poses a
> problem for me.

Periodic streams have long been known (since RIP times) to tend to
synchronize over time.  The OWAMP specification lets you send both
Poisson and periodic streams (and arbitrary other ones), but this
should be taken into consideration.

> Do you think it would be possible within the protocol to _efficiently_
> negotiate a common starting time when both ends have sets of timeslices
> available, but which may differ in most cases? In my opinion a simple
> ACK/NACK type of negotiation could introduce a lot of round-trips for
> that scenario.

The Internet is global.  Having a central authority negotiate sending
times for every test packet doesn't seem feasible.

> I think we'll handle our case of permanent measurements as a special
> case of this. Maybe by specifying a special value of 2^32-1 as the
> number of packets? The implementation should then be able to take care
> of it like I want to see it.

You could also do a sequence of measurement sessions, with the time
between measurement sessions distributed identically to the time
between packets within a test session.

-- 
Stanislav Shalunov		http://www.internet2.edu/~shalunov/

Sex is the mathematics urge sublimated.                 -- M. C. Reed.

_______________________________________________
ippm mailing list
[email-masked] 
https://www1.ietf.org/mailman/listinfo/ippm

---

From: Roland Karch <[email-masked]>
Date: Fri, 04 Feb 2005 19:19:22 +0100

Hello Stanislav,

At 17:51 04.02.2005, stanislav shalunov wrote:
>than the particular implementation.

Actually, the mail exchange was about how to integrate the OWAMP protocol 
into our DFN IPPM implementation. But yes, we can consider it a talk about 
the protocol itself already.

>source for OWAMP implementations.

You won't read any disagreement from me on that matter. While one could of 
course try to develop a completely flexible test protocol, this would most 
likely add too much overhead to the packets. And those should be as small 
as possible...

We've even been playing with the thought of only sending a sequence number 
within each test packet. Theoretically, this bit of information is enough 
to retrieve a complete set of measurement data from both ends.

>times for every test packet doesn't seem feasible.

It of course wouldn't be a central authority for the whole Internet, that 
would indeed be a very ambitious project. However, we do have coordinating 
entities for measurement domains which are used for network surveillance. 
And of course you can set the policies of an OWAMP node in such a way that 
measurement requests will only be granted when coming from that 
coordinating entity.

The interesting question is wether there is (or could be) a protocol hook 
that allows two measurement points to negotiate a common schedule with the 
same (or at least reasonable) efficiency as a coording entity would allow.

>between packets within a test session.

I have a few reasons for handling permanent measurements separately from 
conventional on-demand measurements. If those weren't there, I would gladly 
do them like you suggested. Should you would like to discuss this in 
detail, feel free to pick me up on that point again. I've already had a few 
interesting chats on this matter in the past. :-)

Greetings,
Roland

---

From: "Jeff W. Boote" <[email-masked]>
Date: Fri, 04 Feb 2005 14:53:08 -0700

[Stas and other IPPM list watchers - this thread started out as a discussion on 
what would be needed to get the DFN IPPM OWD implementation to interact with our 
owamp implementation for the purposes of joint measurement between our networks. 
Basically, how much of the OWAMP protocol would need to be implemented within 
IPPM for different levels of interoperability. But, this is not strictly all 
protocol related. Oh, and another confusion. DFN has an OWD application called 
IPPM - so there might be some confusion as to the meaning of the term IPPM.]

Roland Karch wrote:
>>So, for a more concrete example: If you want to send a set of 5 packets every 1 
>>min using the OWAMP control protocol you would specify your send schedule as:
>>
>>slot index	slot type	slot arg
>>0		1(periodic)	60 (1 min)
>>1		1		0
>>2		1		0
>>3		1		0
>>4		1		0
> 
> 

I believe your last sentence demonstrates a different measurement philosophy for 
your IPPM implementation than we have used in the Internet2 owamp implementation.

Our owamp implementation was written with the expectation that it would run on a 
multi-user operating system in user space. This is not a requirement of the 
protocol, and another implementation could get much much better accuracy than we 
do (imagine using a NIC that understood OWAMP and had GPS built-in). The 
advantage of our implementation is that it should be possible to install, run 
and get "reasonable" results on a wide variety of hardware/OS combinations. 
Given this type of implementation, we will have "interesting effects" simply due 
to OS job scheduling.

For these reasons, on the Abilene network we have relied more upon statistical 
views of the owamp data that discount outliers. It sounds like IPPM attempts to 
minimize the "interesting effects" while testing much more than we do. (I 
believe the OWAMP protocol will still work fine for you.)

I bring up this difference only because I think it illustrates why you are so 
much more interested in coordinating your measurements than we have been.

This might give you some insight into our philosophy for Abilene measurements: 
We typically look at the 95th percentile of latency relative to the median 
latency on a given path. So, we are counting on less than 5% of our packets 
experiencing those "interesting effects". If more than 5% of the packets in a 
given time period deviate significantly from the median, than we have usually 
found queuing. We usually look at 30 second samples and we typically send 10 
packets per second per path of interest. So, we are usually looking at 300 
packet timings. On Abilene that equates to a minimum of 22 sending streams and 
and 22 receiving streams on each one of the Abilene owamp hosts with additional 
streams for tests with peer networks and institutions.

>>You could also specify slot type 0 for slot index 0 if you wanted to send those 
>>5 packets close to one per minute on a Poisson distribution so you can avoid 
>>periodic events on the network.
> 
> 

I don't think you would want to do this within the protocol. What I would 
probably think of instead is to have an entity that coordinated the tests and 
used an Identity/AES key to identify them. That set of Identity/AES keys and 
coordinated schedule definitions could then be distributed to the "back-end" of 
your servers. Then, you could use the authenticated mode of OWAMP to request a 
test that you have already "scheduled" to actually run it.

>>The other part of the test request indicating the schedule would be to specify 
>>the number of packets you wanted in the full test stream. (We make our tests 
>>extremely long so we don't have to speak OWAMP-control very often. On the order 
>>of 24 hours.) For example, if you specified 1000 for the number of packets in 
>>the test, then this test would take 200 minutes to finish. 5 packets would be 
>>sent every minute. This is all described in section 3.6 of the spec (page 17).
> 
> 

I don't think adding special values into the test request (for the general case) 
is such a good idea because of interoperability. It is possible that another 
implementation could actually use 2^32-1 to request a test and actually mean 
that many packets. After all, that is the maximum number of packets that can be 
requested and depending upon how much bandwidth you allow, that doesn't have to 
be that long of a time period on a 10 GigE interface... Granted, this might not 
be the best way to get good latency information but it would be a reasonable way 
to use OWAMP to look for line-rate reordering.

The out-of-band trick I suggested above would work for this as well, while 
keeping interoperability. You could share a specific "identity" name out of 
bounds of the OWAMP protocol. (There is no way to distribute identities within 
the protocol anyway.) But the point is that your implementation could assign 
specific meaning to this identity. You could say that anything that 
authenticates with this identity means "infinity" if it specifies 2^32-1. (Or as 
indicated above, anything that authenticates with this identity has already 
allocated a specific schedule.)

jeff

---

From: Sven Ubik <[email-masked]>
Date: Mon, 7 Feb 2005 10:19:29 +0100 (CET)

Hi all,

one implementation question: does IPPM and OWAMP currently support
IPv6 and multicast? I can imagine that some performance characteristics
(such as delay fluctuation, from what is in the scope of IPPM and OWAMP)
can be different between IPv4 and IPv6 and between unicast and multicast
due to differences in packet processing in routers.

Regards,

Sven Ubik

---

From: Roland Karch <[email-masked]>
Date: Mon, 07 Feb 2005 11:13:25 +0100

> IPPM - so there might be some confusion as to the meaning of the term IPPM.]

We're trying to work on that second part since a few weeks. Hopefully,
we'll have an innovative name soon. Till then, we at least try to avoid
ambiguity by calling it DFN IPPM.

> do (imagine using a NIC that understood OWAMP and had GPS built-in).

Indeed, a specialized NIC can help tremendously. We also made some
interesting measurements [1] with the Endace cards. However, it would be
almost impossible to build a tight mesh of such specialized equipment.

> The 

Yes, one of our prime goals is to reach an as high as possible accuracy.
If you're interested in that particular phenomenon, in [2] you can see
the impact of such a "packet train" configuration without minimal wait
states after sending each packet. On the left and right borders of the
diagram, the wait states were in place, while they were missing for the
bigger part of the measurement. The difference is actually no big deal,
however it's easier to spot real network phenomena when you have a sharp
baseline for the measurements.

> streams for tests with peer networks and institutions.

That makes for quite a fascinating comparison between our measurement
networks, indeed. So while DFN has substantially more single
measurements (almost 1300) we seem to have substantially less
measurement traffic per host (5 packets every 30s per measurement path).
I'm looking forward to seeing how the measurements will perform when
directly colocated.

> test that you have already "scheduled" to actually run it.

That works well for the case of a central entity. While I am actually
someone who advertises this solution, I would still enjoy to actually
live without it. But I guess that this would add a lot of unwanted
complexity to the negotiation protocol?

> allocated a specific schedule.)

Thanks for these hints, I think they'll work nicely in practice.

Greetings,
Roland

[1]
http://www.win-labor.dfn.de/ippm/passivmessungen.html
[2]
http://www.win-labor.dfn.de/cgi-bin/ipqos/ipqos_detail.pl?date=04.02.2005&metric=owd&from=ippm_trimble_m&to=Frankfurt_DFN&psize=429&ymin=15&ymax=20&ylimit=on&begin=09:00:00&end=14:10:00
_______________________________________________
ippm mailing list
[email-masked] 
https://www1.ietf.org/mailman/listinfo/ippm

---

From: Henk Uijterwaal <[email-masked]>
Date: Mon, 07 Feb 2005 12:22:46 +0100

>due to differences in packet processing in routers.

The metrics developed by the IPPM group are independent of the IP protocol
version used, it is up to the implementors to decide if they want to do
measurements on IPv4 or IPv6 (or both).  Of the 5 implementations that
are mentioned in the implementation report, (at least) 2 support IPv6,
all 5 support IPv4.

The OWAMP protocol as such supports IPv4 and IPv6, again leaving it up to
the implementors to decide which versions are supported in their 
implementation.

There has been some discussion on the mailing list on Multicast
related metrics, and there is an individual submission draft on
multicast metrics (draft-liang-multiparty-para-00.txt).  At the moment,
the WG is discussing if/what we should do in this area.  If you have
any thoughts, speak up.

Henk (IPPM WG co-chair).

------------------------------------------------------------------------------
Henk Uijterwaal                           Email: henk.uijterwaal(at)ripe.net
RIPE Network Coordination Centre          http://www.amsterdamned.org/~henk
P.O.Box 10096          Singel 258         Phone: +31.20.5354414
1001 EB Amsterdam      1016 AB Amsterdam  Fax: +31.20.5354445
The Netherlands        The Netherlands    Mobile: +31.6.55861746
------------------------------------------------------------------------------

Look here junior, don't you be so happy.
And for Heaven's sake, don't you be so sad.                 (Tom Verlaine)

---

From: stanislav shalunov <[email-masked]>
Date: 07 Feb 2005 11:21:00 -0500

Sven Ubik <[email-masked]> writes:

> [...] does IPPM and OWAMP currently support IPv6

Yes.  OWAMP can do either IPv6 and IPv4.  You can even have the
control connection and the test stream use different protocol numbers.

> and multicast?

No.  The IPPM WG, so far, hasn't defined any multicast-related metric
that a protocol such as OWAMP could help measure.

This message is designed to be viewed at an angle of 45 degrees.

---

From: "Jeff W. Boote" <[email-masked]>
Date: Mon, 07 Feb 2005 10:32:19 -0700

> due to differences in packet processing in routers.

The Internet2 owamp implementation supports IPv4 and IPv6. The owamp 
implementation does not support mulitcast. The OWAMP control protocol itself was 
designed for unicast, not multicast. It would be a non-trivial exercise to 
extend it for multicast use.

However, I believe the OWAMP-Test packet format would still be beneficial and 
appropriate for multicast latency measurements.

jeff