Subject: Re: [ippm] metric registry url and related format / automation question

From: "MORTON, ALFRED C (AL)" <[email-masked]>
Date: Mon, 12 Oct 2015 09:31:08 -0400

Hi Juergen,
I somehow missed this one -- reply to your comment below.
Al
> -----Original Message-----
> From: ippm [mailto:[email-masked]] On Behalf Of Juergen
> Schoenwaelder
> Sent: Wednesday, September 30, 2015 6:38 AM
> To: [email-masked]
> Subject: [ippm] metric registry url and related format / automation
> question
> 
> Hi,
> 
> every registry entry has a set of identifiers:
> 
> - a number (registry unique)
> - a (longish) name (registry unique)
> - a URN (a concatenation of the name with a registry prefix) (globally
> unique)
> - a URL to point to something retrivable
> 
> The last one seems rather underspecified. Is the URL pointing to a web
> page that is intended for humans to read about the metric 
[ACM] 
Yes, there are several reasons to do this:

We will have a URL pointing to exactly *one* registry entry. 
I discussed this possibility with IANA, and they (Amanda) 
agreed to provide it.

The separate files can be more easily edited and re-used
by folks when preparing new entries.

The separate files can be copied and handed to a developer, 
"augment the code so it performs this metric". 
An HTML file could have links to all the references the developer needs,
like the HTML-versions of Internet Drafts (and the code that generates
them).

We have a registry with 4 Summary columns, with the "long-ish name"
and Description. when you add the nitty-gritty details there are
about 20 columns and some of the entries will have a lot of text. 
A table with ~20 columns, some wide, would be difficult to read
on the screen.

> ...or is the idea
> that I get a machine readable version of the registry entry? In the
> later case, the format would have to be defined somehow. 
[ACM] 
No. I think we still need to learn a solid use case 
and the specific columns that need to be machine parse-able.
The Fixed parameters column is certainly a candidate based on 
discussions so far, but how and when in the measurement system
development process/operation would it be better to have machine 
parse-able info? Many of our columns contain info that machines 
would find useless (e.g., Description). 

> ... I guess the
> underlying question is whether tools are expected to access the registry
> in order to for example dynamically learn about the possible parameters
> (and their data types) for a metric or is the idea that essentially the
> conversion of the registry content into running code is a manual
> process?
[ACM] 
The development of measurement code is certainly a manual process,
and those developers are our target users.

I think the over-arching goal of our registry work is to ensure
that independent implementations of measurement systems could
conduct measurements of the same metrics using equivalent methods,
and produce comparable results -- IOW, two implementations measuring
identical network conditions would produce statistically equivalent
results.

The idea that measurement devices would access any version of registry
as part of on-going operations never crossed my mind. The possibility
was something Barbara first mentioned in Dallas this year (and that's
over two years after some of us started work to define the registry).

So, it's certainly possible to produce some machine-readable formats.
Or, for humans other than the registry authors to produce them and
make them accessible to their measurement system (which could be operating
in a private address space where the IANA registry is un-reachable).

What developers plan such dynamic systems that would access the registry
on an on-going basis?

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Sun, 18 Oct 2015 11:21:59 +0200

Al,

I got a bit lost in the answer. So is the URL resolving to something
that has some structured data or does it refer to a web page rendered
for human readers? I do not think that measurement tasks residing on
MAs are likely to read registry information but controllers and
applications consuming data pushed to collectors might want to do so.

/js

PS: Yes, thinking about a registry entry as a big table row is scary.
    I would prefer to have an IANA managed table with just the keys
    for the registered metrics (number, name, URN, URL) where the URL
    points to a structured resources that allows tools to process the
    various fields that make up the registered metric entry.

-- 
Juergen Schoenwaelder           Jacobs University Bremen gGmbH
Phone: +49 421 200 3587         Campus Ring 1 | 28759 Bremen | Germany
Fax:   +49 421 200 3103         <http://www.jacobs-university.de/>

---

From: "MORTON, ALFRED C (AL)" <[email-masked]>
Date: Sun, 18 Oct 2015 09:59:18 -0400

Hi Juergen,

The URL points to a web page that is formatted for humans to read.*

I'm sorry to hear that adding the rationale behind this choice
made it less useful to you, though I thought that the "Yes" and "No"
answers to your question describing the human/machine alternatives 
were unambiguous, at least. :-)  And, I'm glad we agree it's 
necessary to avoid the super-wide table of entries. This is an
aspect I've been concerned about (and even mocked-up for my 
co-authors to help make my case for URLs a year ago! as we've
both re-experienced recently: examples help understanding).

>>> WGLC <<<
I also see that we failed to make the "human-readable" aspect clear
in the current text of 
https://tools.ietf.org/html/draft-ietf-ippm-metric-registry-05#section-7.1.3
so, this is a WGLC comment: Please add the first sentence of this message
and possibly the footnote below in section 7.1.3.
>>> /WGLC <<<

you wrote:
> ... I do not think that measurement tasks residing on MAs
> are likely to read registry information but controllers and applications
> consuming data pushed to collectors might want to do so.

Thanks for expressing your perspective! It begins to define the
Use Case(s) for structured data that machines can easily parse.
There's much more input required before we understand the needs,
but if we can agree, then also having a URL to a 
"machine-parse-able" file could be an optional part of each 
Registered Metric.

Thanks again for all your comments, especially on:
https://tools.ietf.org/html/draft-morton-ippm-initial-registry-00
(which will be updated shortly).

regards,
Al

* The ideal form of this page is very similar to our HTML-ized
  Internet Drafts, where each RFC/Section# reference is a 
  hyperlink to the needed material. 

> From: Juergen Schoenwaelder [mailto:j.schoenwaelder@jacobs-
> university.de]
> Sent: Sunday, October 18, 2015 5:22 AM
> To: MORTON, ALFRED C (AL)
> Cc: [email-masked]
> Subject: Re: [ippm] metric registry url and related format / automation
> question
> 
> Al,
> 
> for human readers? I do not think that measurement tasks residing on MAs
> are likely to read registry information but controllers and applications
> consuming data pushed to collectors might want to do so.
> 
> /js
> 
> > > The last one seems rather underspecified. Is the URL pointing to a
> > > web page that is intended for humans to read about the metric
> > [ACM]
> > I discussed this possibility with IANA, and they (Amanda) agreed to
> > provide it.
> >
> > The separate files can be more easily edited and re-used by folks when
> > preparing new entries.
> >
> > The separate files can be copied and handed to a developer, "augment
> > the code so it performs this metric".
> > An HTML file could have links to all the references the developer
> > needs, like the HTML-versions of Internet Drafts (and the code that
> > generates them).
> >
> > and Description. when you add the nitty-gritty details there are about
> > 20 columns and some of the entries will have a lot of text.
> > A table with ~20 columns, some wide, would be difficult to read on the
> > screen.
> >
> >
> > > ...or is the idea
> > No. I think we still need to learn a solid use case and the specific
> > columns that need to be machine parse-able.
> > parse-able info? Many of our columns contain info that machines would
> > find useless (e.g., Description).
> >
> > > ... I guess the
> > > underlying question is whether tools are expected to access the
> > > registry in order to for example dynamically learn about the
> > > possible parameters (and their data types) for a metric or is the
> > > idea that essentially the conversion of the registry content into
> > > running code is a manual process?
> > [ACM]
> > The development of measurement code is certainly a manual process, and
> > those developers are our target users.
> >
> > I think the over-arching goal of our registry work is to ensure that
> > independent implementations of measurement systems could conduct
> > measurements of the same metrics using equivalent methods, and produce
> > comparable results -- IOW, two implementations measuring identical
> > network conditions would produce statistically equivalent results.
> >
> > make them accessible to their measurement system (which could be
> > operating in a private address space where the IANA registry is un-
> reachable).
> >
> > What developers plan such dynamic systems that would access the
> > registry on an on-going basis?
> >
> 
> --