Subject: Re: [ippm] next steps with IPPM registry

From: Juergen Schoenwaelder <[email-masked]>
Date: Tue, 1 Dec 2015 11:04:09 +0100

On Tue, Dec 01, 2015 at 06:39:31AM +0100, marcelo bagnulo braun wrote:
> Hi,
> 
> I understand that the only remaining open issue with the registry is to 
> figure out if/how we want to include machine readable information in it.
> 
> In order to figure this out, i understand the idea was for proponents of 
> doing so should explain the use cases they have in mind, so we can 
> understand if we want to do this and now.
> 
> I understand Barbara and Juergen were proposing this, so could you (or 
> anyone else who thinks this is a good idea) to explain the use cases?
>

I think the question is which role the registry plays in the overall
framework. Is it (i) primarily some form of documentation an MA can
refer to or (ii) is the idea that the registry is normative when it
comes to run-time parameters and data formats. In the later case, I
think it is desirable that tools can work with the portions of the
registry that are used by implementations in order to drive automation
(and this likely also requires putting restrictions on say parameter
names - for example, the parameter name '1/lambda' in
draft-morton-ippm-initial-registry-01.txt is likely troublesome if you
take it literally in implementations).

Perhaps the answer is in between, namely that the registry is
primarily documentation for humans and that a formalization of the
parameters that are relevant for implementations is a separate step
(e.g., for metric X in the registry, some human writes a concrete YANG
module defining the concrete configuration parameters to configure
metric X following the guidelines given in the registry).

If the registry is primarily documentation (and this is what section 5
might be saying), then I would prefer if parameters are described with
their semantics and units and precision but not the specific encoding.

/js

-- 
Juergen Schoenwaelder           Jacobs University Bremen gGmbH
Phone: +49 421 200 3587         Campus Ring 1 | 28759 Bremen | Germany
Fax:   +49 421 200 3103         <http://www.jacobs-university.de/>

---

From: "MORTON, ALFRED C (AL)" <[email-masked]>
Date: Tue, 1 Dec 2015 11:41:59 -0500

Hi Juergen and Marcelo (and Brian Trammell),
please see more discussion below,
Al [ACM]

> -----Original Message-----
> From: ippm [mailto:[email-masked]] On Behalf Of Juergen
> Schoenwaelder
> Sent: Tuesday, December 01, 2015 5:04 AM
> To: marcelo bagnulo braun
> Cc: STARK, BARBARA H; [email-masked]
> Subject: Re: [ippm] next steps with IPPM registry
> 
> > I understand that the only remaining open issue with the registry is
> > to figure out if/how we want to include machine readable information
> in it.
> >
> > In order to figure this out, i understand the idea was for proponents
> > of doing so should explain the use cases they have in mind, so we can
[ACM] 
The registry entries are certainly normative w.r.t. run-time parameter
semantics and units and precision. By providing a data format, we easily
achieve the requirements for units and precision, and ensure the values 
appear in a form which can be consumed in many measurement protocols
without conversion. Since many of the values are needed in binary format,
the tools should take the role of accepting human-readable input and 
conversion to the binary protocol fields (and display both when necessary).
This applies to both Fixed and Run-time parameters, and also to 
output results. 

The above description of consumable formats for measurement protocols 
is a viable use case for the data formats currently provided in 
draft-morton-ippm-initial-registry.

So, I would like to hear a use case for tools working with portions of 
the registry. What (LMAP) functional entities use these tools, 
and how does registry access make their job easier? 

> names - for example, the parameter name '1/lambda' in draft-morton-ippm-
> initial-registry-01.txt is likely troublesome if you take it literally
> in implementations).
[ACM] 
During the IPPM-94 session, Brian Trammell commented on this, too (from the minutes):

"Brian wondered about using names which could be automatically converted using APIs. 
Al said that there was some discussion about hierarchical naming. 
[ACM - this was during the RAIM workshop].
Will talk about this offline."

Specifications for naming are new to me, I would be glad to try to use
someone's suggestion if it helps.

> 
> Perhaps the answer is in between, namely that the registry is primarily
> documentation for humans and that a formalization of the parameters that
> are relevant for implementations is a separate step (e.g., for metric X
> in the registry, some human writes a concrete YANG module defining the
> concrete configuration parameters to configure metric X following the
> guidelines given in the registry).
[ACM] 
So, the registry was never intended to replace a YANG model for its
ability to convey configuration information, 
and it must contain information that should never appear in a YANG model,
that is information for measurement implementers to attain the goal
of measurements that can be compared between independent implementations.

> their semantics and units and precision but not the specific encoding.
[ACM] 
But we do have a use case for the encoding formats, described above.
After specifying the encoding, I would be willing to add:
"... or alternate format for values with equivalent units and precision,
recognizing that conversion to the specified format may be required."

> _______________________________________________
> ippm mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/ippm

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Tue, 1 Dec 2015 18:09:37 +0100

> This applies to both Fixed and Run-time parameters, and also to 

Configuration interfaces (whether its CLIs or YANG-based interfaces)
usually do not operate on binary data. Are we talking about parameters
that get configured or not?

> and how does registry access make their job easier?

I may not understand your question. I am starting form an LMAP YANG
data model and if I use a certain metric, I believe I have to
configure certain parameters. In some legislations, customers are free
to deploy CPEs of their choice and hence it would be nice if tools can
discover parameters by discovering the capabilities of an LMAP agent
and then following the pointer to a registry that defines the
parameters. But it could also be that the registry is not supposed to
support tools driving automation (e.g. tools able to display
parameters that can be set without having a human programmer to
program this for every device in the field).

> someone's suggestion if it helps.

Having parameter names that start with a digit and that look like an
expression may cause a lot of fun. Usually parameter names are
restricted much like variable names in many programming languages. If
the parameter names indeed are normative, I think we should opt for
names that have a high chance to work across many programming and data
modeling languages. To make a concrete proposal: Limit parameter names
to [a-z,A-Z,_]+ or something like that.

> of measurements that can be compared between independent implementations.

I never proposed to write the registry in YANG. The parameters,
apparently, are an interface between the registry and tha YANG model
so we should understand how this works in practice, in particular if
the parameters are normative.

> recognizing that conversion to the specified format may be required."

But see, I also have a use case (LMAP) where binary encodings are not
used. Perhaps the easiest solution is to find a loose coupling since a
tighter coupling apparently creates some tension. That is, to drive
automation, we would rely on a human created machine readable
serialization of data model content, not the registry itself. In other
words, the registry pointer really is just a reference for
documentation.

/js

---

From: "Weil, Jason" <[email-masked]>
Date: Tue, 1 Dec 2015 17:10:12 +0000

Marcelo,

I was talking with Al after the IPPM meeting in Yokohama about the need to
consider how IP version number may impact the metric registry and initial
registry entries. Barring the fact that we still need to update RFC2330
for IPv6, I believe we may need to tweak the IPPM registry to account for
the differences between the IPv6 and IPv4 Headers. This seems to be
primarily an issue for the Fixed Parameter values. The Run-time Parameters
descriptions currently include support for both IPv4 and IPv6 values, but
this too may need to change depending on how fixed the Fixed values need
to be.

For example, Section 4 in draft-morton-ippm-initial-registry-01.txt
describes the registry entry for the UDP Round-trip Latency metric.
Subsection 4.2.2 describes the Fixed Parameters for the metric part of
which is the following Type-P parameters:

o  IPv4 header values:

  *  DSCP: set to 0

  *  TTL: set to 255

  *  Protocol: Set to 17 (UDP)

The DNS Response Latency Registry Entry (Section 6.2.2) has a similar
issue:

   Type-P:

   o  IPv4 header values:

  *  DSCP: set to 0

  *  TTL set to 255

  *  Protocol: Set to 17 (UDP)

So if we go down this path we will end up with two metric entries, 1 for
IPv4 and one for IPv6 with separate Fixed parameters depending on the IP
version number. If we do this I think we should also add the IP version
number into the metric name as well to help distinguish the metric entries
in the registry.

I guess another option would be to include both the IPv4 header and IPv6
header in the Fixed Parameters fields but that would seem to increase the
logic needed by the system reading the metric which may hinder or at least
impact agent interoperability.

-Jason

On 12/1/15, 12:39 AM, "ippm on behalf of marcelo bagnulo braun"
<[email-masked] on behalf of [email-masked]> wrote:

>Thanks, marcelo

________________________________

This E-mail and any of its attachments may contain Time Warner Cable proprietary information, which is privileged, confidential, or subject to copyright belonging to Time Warner Cable. This E-mail is intended solely for the use of the individual or entity to which it is addressed. If you are not the intended recipient of this E-mail, you are hereby notified that any dissemination, distribution, copying, or action taken in relation to the contents of and attachments to this E-mail is strictly prohibited and may be unlawful. If you have received this E-mail in error, please notify the sender immediately and permanently delete the original and any copy of this E-mail and any printout.

---

From: "MORTON, ALFRED C (AL)" <[email-masked]>
Date: Mon, 14 Dec 2015 09:04:04 -0500

Although this is not the last word by any means, 
I offer this revised draft for further discussion (probably in 2016):
https://tools.ietf.org/html/draft-morton-ippm-initial-registry-02

A diff from the previous version is available at:
https://www.ietf.org/rfcdiff?url2=draft-morton-ippm-initial-registry-02

Again, the work to address comments has been restricted to 
Section 4 - other sections will follow when we have more agreement.

One comment was on a section with IPv4 only, v6 is there now.
There's also some work on Parameter names, using Jürgen's suggestions.

I agree to use other data formats where they are available
and where they are exact. This seems to leave us with a gap:
representing high precision time values without help from
the NTP formats - in fact, the NTP 32-bit format may not have
sufficient resolution for some measurements. I seek alternatives
for this purpose, and your suggestions.

One further point: The vision for registry users is currently three-fold:

1. Measurement function developers can look at the registry entries
(where the Measurement function is what the 
LMAP MA talks to, it's not one of the LMAP functions.)
and know exactly what they must construct and support in terms of
methods and input variables. This is critical information, because 
it is difficult to design a system that performs its task accurately
when the range of operation is very wide and unknown - IOW, the 
input parameters could be set to unexpected values (our Fixed Parameters).
This was one driving force behind *fixing all possible parameters*,
another is to have comparable measurement results from different systems.
OTOH, developers must expect wide variation on Run-time parameters,
so we make the distinction between them.  The Fixed parameters are 
hard-coded in the implementation of a given metric/measurement. 

2. The LMAP protocols designers can request a specific metric to be 
measured by signaling the Registry Entry ID and any Run-time Parameters required.
Therefore, the format of the Run-time parameters and the Output results
will be exchanged among LMAP entities and the Measurement function.
Having these formats specified in the Registry will help avoid 
interoperability issues. 

3. People conducting measurement studies can task their suppliers to
provide Measurement functions and other system components (LMAP) that
provide measurement support for a list of Registered Metrics. 
Multiple suppliers can be deployed because all systems make measurements
exactly the same way.

If there are other uses/users for the Registry, let's list them.

thanks and regards,
Al

> From: Juergen Schoenwaelder [mailto:j.schoenwaelder@jacobs-
> university.de]
> Sent: Tuesday, December 01, 2015 12:10 PM
> To: MORTON, ALFRED C (AL)
> Cc: marcelo bagnulo braun; STARK, BARBARA H; [email-masked]
> primarily

---

From: "MORTON, ALFRED C (AL)" <[email-masked]>
Date: Mon, 14 Dec 2015 09:24:52 -0500

Hi Jason,
My proposed solution for simple registry entries is to allow for
the possibility of either IPv4 or IPv6.  Anything more complicated,
like mixed address families or v6 Options, would need its own
entry. As I mentioned:
https://tools.ietf.org/html/draft-morton-ippm-initial-registry-02
does the simple thing in section 4.

Thanks for your comment,
Al

> From: ippm [mailto:[email-masked]] On Behalf Of Weil, Jason
> To: marcelo bagnulo braun; [email-masked]; STARK, BARBARA H; Juergen
> Schoenwaelder
> of
> This E-mail and any of its attachments may contain Time Warner Cable
> proprietary information, which is privileged, confidential, or subject
> to copyright belonging to Time Warner Cable. This E-mail is intended
> solely for the use of the individual or entity to which it is addressed.
> If you are not the intended recipient of this E-mail, you are hereby
> notified that any dissemination, distribution, copying, or action taken
> in relation to the contents of and attachments to this E-mail is
> strictly prohibited and may be unlawful. If you have received this E-
> mail in error, please notify the sender immediately and permanently
> delete the original and any copy of this E-mail and any printout.
>