Subject: [lmap] LMAP Framework issue #3 (take 2) Interactions between MA and Controller

From: <[email-masked]>
Date: Thu, 12 Sep 2013 16:44:34 +0100

Thanks for the nice discussion, I've tried to re-write to reflect where I think we've got to. I also changed the title to:

Interactions between MA and Controller
As part of a bootstrap process the MA gets sufficient information to contact a Controller. Defining a bootstrap mechanism is out of scope of the LMAP WG.

The basic interaction between a Controller and MA is that the Controller sends an Instruction to the MA, detailing what Measurement Tasks to do, when to do them, and how to report the Measurement Results. In general we expect that the measurement system understands what Measurement Methods the MA can do and therefore the Controller can simply instruct the MA; there is no need for a negotiation protocol (which would add complexity to the MA, Controller and Control Protocol). However, it is possible that the MA is in fact not capable of performing the requested Measurement Method, and so the Control Protocol must allow the MA to reply with a suitable error code.

The MA can send to the Controller the list of Measurement Methods that it is capable of. This could be useful:
[1] as part of the MA registering with the Controller. Note that in some scenarios it is not needed, as the Controller will know the information by other means, for example as part of the bootstrapping process (using Tr-069 say) or because the MA is statically configured.
[2] so the MA can re-register, for example if it becomes capable of a new Measurement Method.
** Comment: suggest we keep it simple and just send the complete list, rather than a delta. The message should be short, since Methods are referred to via a registry.
[3] when requested by the Controller, which could be useful if 'something goes wrong' and the Controller forgets what the MA can do.

Note that the advert contains the list of Measurement Methods that the MA can perform. It is not intended to indicate dynamic capabilities like the MA's currently unused CPU, memory or battery life.
** Comment: I'm not sure whether we reached consensus on this point.

It is possible that later, when a MA is implementing the Instruction, it does not perform a particular Measurement Task for some reason, such as: the Measurement Peer is busy, or there is cross-traffic, ... The MA doesn't inform the Controller, instead it is reported to the Collector as part of the Measurement Report. The Collector may in turn tell the measurement system or even the Controller, but this is out of scope of LMAP.
** Comment: are there any conditions when it's critical for the Controller to be informed?

From: [email-masked] [mailto:[email-masked]] On Behalf Of [email-masked]
Sent: 05 September 2013 11:13
To: [email-masked]
Subject: [lmap] LMAP Framework issue #3 No negotiation between MA and Controller

Another issue:
There is no negotiation between the MA and Controller - the Controller simply sends the Instruction to the MA, with the details of the Measurement Tasks to perform.
In general we expect that the measurement system understands the capabilities of its MAs (probably there are only one or a few classes of MA), so negotiation about the MA's capabilities would have little benefit. It would also add complexity to the MA, Controller and Control Protocol.

It is possible that the MA can't perform the requested Measurement Task. So the Control Protocol needs to allow the MA to reply with an error code. It has also been suggested that the Controller should be able to ask the MA to send a list of all the Measurement Methods that it is capable of, in case 'something goes wrong' and the Controller forgets what the MA can do. Comment: this idea hasn't had much discussion yet, but appears in the Information Model i-d

Thoughts?
Thanks
phil

---

From: "Sharam Hakimi" <[email-masked]>
Date: Thu, 12 Sep 2013 12:16:31 -0400

Phil,

Great caption, couple of suggestions:

Change:

As part of a bootstrap process the MA gets sufficient information to
contact a Controller.

To: 

Either as part of a bootstrap process or through pre-configuration the
MA will have sufficient information to contact a Controller.

As for the last question: are there any conditions when it's critical
for the Controller to be informed?

Unless there is a defined critical test time, not that I think one is
necessary, that a test needs to have a set of results, I do not think
that the controller would need to inform the controller that a test
could not be performed. I think that would add a lot of unnecessary
complexity.

Thanks,

Sharam

From: [email-masked] [mailto:[email-masked]] On Behalf Of
[email-masked]
Sent: Thursday, September 12, 2013 11:45 AM
To: [email-masked]
Subject: [lmap] LMAP Framework issue #3 (take 2) Interactions between MA
and Controller

Thanks for the nice discussion, I've tried to re-write to reflect where
I think we've got to. I also changed the title to:

As part of a bootstrap process the MA gets sufficient information to
contact a Controller. Defining a bootstrap mechanism is out of scope of
the LMAP WG.

The basic interaction between a Controller and MA is that the Controller
sends an Instruction to the MA, detailing what Measurement Tasks to do,
when to do them, and how to report the Measurement Results. In general
we expect that the measurement system understands what Measurement
Methods the MA can do and therefore the Controller can simply instruct
the MA; there is no need for a negotiation protocol (which would add
complexity to the MA, Controller and Control Protocol). However, it is
possible that the MA is in fact not capable of performing the requested
Measurement Method, and so the Control Protocol must allow the MA to
reply with a suitable error code. 

The MA can send to the Controller the list of Measurement Methods that
it is capable of. This could be useful:

[1] as part of the MA registering with the Controller. Note that in some
scenarios it is not needed, as the Controller will know the information
by other means, for example as part of the bootstrapping process (using
Tr-069 say) or because the MA is statically configured.

[2] so the MA can re-register, for example if it becomes capable of a
new Measurement Method. 

** Comment: suggest we keep it simple and just send the complete list,
rather than a delta. The message should be short, since Methods are
referred to via a registry.

[3] when requested by the Controller, which could be useful if
'something goes wrong' and the Controller forgets what the MA can do. 

Note that the advert contains the list of Measurement Methods that the
MA can perform. It is not intended to indicate dynamic capabilities like
the MA's currently unused CPU, memory or battery life. 

It is possible that later, when a MA is implementing the Instruction, it
does not perform a particular Measurement Task for some reason, such as:
the Measurement Peer is busy, or there is cross-traffic, ... The MA
doesn't inform the Controller, instead it is reported to the Collector
as part of the Measurement Report. The Collector may in turn tell the
measurement system or even the Controller, but this is out of scope of
LMAP.

** Comment: are there any conditions when it's critical for the
Controller to be informed?

From: [email-masked] [mailto:[email-masked]] On Behalf Of
[email-masked]
Sent: 05 September 2013 11:13
To: [email-masked]
Subject: [lmap] LMAP Framework issue #3 No negotiation between MA and
Controller

Another issue:

There is no negotiation between the MA and Controller - the Controller
simply sends the Instruction to the MA, with the details of the
Measurement Tasks to perform. 

In general we expect that the measurement system understands the
capabilities of its MAs (probably there are only one or a few classes of
MA), so negotiation about the MA's capabilities would have little
benefit. It would also add complexity to the MA, Controller and Control
Protocol. 

It is possible that the MA can't perform the requested Measurement Task.
So the Control Protocol needs to allow the MA to reply with an error
code. It has also been suggested that the Controller should be able to
ask the MA to send a list of all the Measurement Methods that it is
capable of, in case 'something goes wrong' and the Controller forgets
what the MA can do. Comment: this idea hasn't had much discussion yet,
but appears in the Information Model i-d

Thoughts? 

Thanks

phil

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Thu, 12 Sep 2013 19:01:00 +0200

> MA will have sufficient information to contact a Controller.

For me, pre-configuration is one way of bootstrapping. Hence I believe
the original text was sufficient and the addition is not needed.

/js

-- 
Juergen Schoenwaelder           Jacobs University Bremen gGmbH
Phone: +49 421 200 3587         Campus Ring 1, 28759 Bremen, Germany
Fax:   +49 421 200 3103         <http://www.jacobs-university.de/>

---

From: "STARK, BARBARA H" <[email-masked]>
Date: Thu, 12 Sep 2013 18:34:24 +0000

First a process question:
How detailed is this Framework document supposed to be? Is it intended to include the full list of criteria for a Control Protocol, or will these criteria be documented separately? If the first, then I would prefer for the criteria to be more precisely described, and easy to identify as Control Protocol criteria/requirements. If the latter, then I would prefer for the Framework to use more descriptive language as to what is expected of the Control Protocol, rather than trying to give names to all of the various expected functionality (e.g., Instruction, registration). More detailed comments below.
Barbara

From: [email-masked] [mailto:[email-masked]] On Behalf Of [email-masked]
Sent: Thursday, September 12, 2013 11:45 AM
To: [email-masked]
Subject: [lmap] LMAP Framework issue #3 (take 2) Interactions between MA and Controller

<bhs> The use of the word "Instruction" here makes me uncomfortable, because it implies (at least to me) a manner of communication that is not quite consistent with the way most WAN-based massive-number-of-controlled-devices control protocols work today. "Instruction" suggests a manner of communication where the desired action is a part of the basic message. Having separate message syntax (actions) for everything the Controller wants the MA to do would greatly increase the number of messages that need to go back and forth between MA and controller. This sort of interaction is common in protocols intended to work on LAN segments. It's a very bad idea for protocols intended to work over the WAN that are intended to scale to large numbers of devices, and be highly extensible. I would prefer if we did not attempt to give this function of the Controller to MA interaction a proper name (denoted by a capital first letter). I would also prefer if we described it as "the Controller configures the MA", using the word "configure" instead of "instruct".

I don't think we've defined "negotiation protocol". Probably because we don't need it. I would prefer if we got rid of that (negotiation protocol) sentence and just used the last sentence as a positive statement about what we do expect from the Control Protocol. For example: The Control Protocol must support robust error reporting by the MA, to provide the Controller with sufficiently-detailed reasons for any failures in configuration.

Is "measurement system" defined? I'm a bit unclear on what this is.

Resulting recommended text:
The basic interaction between a Controller and MA is that the Controller configures the MA, detailing what Measurement Tasks to do, when to do them, and how to report the Measurement Results. In general we expect that the Controller knows what Measurement Methods the MA supports, such that the Controller can correctly configure the MA. However, the Control Protocol must support robust error reporting by the MA, to provide the Controller with sufficiently-detailed reasons for any failures in configuration.

<bhs> The word "register" also makes me uncomfortable, though not as much as "Instruct". This implies to me some very specific functionality than I don't consider completely necessary. But maybe if "registration" were defined I would feel better about it. The "**Comment" suggestion is too specific for the Framework, IMO. But if Framework has detailed Control Protocol criteria, then I would reword the statement as "The Control Protocol must allow the Controller to request the MA supply the MA's complete list of supported Measurement Methods, in a concise format."  I would prefer if the above statements were reworded as:
The MA can send to the Controller the list of Measurement Methods that it is capable of. This could be useful:
[1] when the MA first communicates with a Controller.
[2] when the MA becomes capable of a new Measurement Method.
[3] when requested by the Controller (e.g., if the Controller forgets what the MA can do or otherwise wants to resynchronize what it knows about the MA).

<bhs> I would have no objection to including such capabilities as optional elements of the information model. In fact, I think it would be a good idea. But I don't think this needs to be mentioned in the Framework.
It is possible that later, when a MA is implementing the Instruction, it does not perform a particular Measurement Task for some reason, such as: the Measurement Peer is busy, or there is cross-traffic, ... The MA doesn't inform the Controller, instead it is reported to the Collector as part of the Measurement Report. The Collector may in turn tell the measurement system or even the Controller, but this is out of scope of LMAP.
** Comment: are there any conditions when it's critical for the Controller to be informed?

<bhs> Ditto my previous objection to "Instruction". I believe the information model needs to include status parameters that would allow the Controller to request this info, at some level. This can be optional to support for MA and Controller. I recommend rewording to:
It is possible that an MA is unable to carry out a configured Measurement Task. Possible reasons include the Measurement Peer being busy, presence of cross-traffic, ... The MA will report this information to the Collector as part of the Measurement Report. The Collector may in turn tell other systems (including, possibly, the Controller), but this is out of scope of LMAP. The MA may also be able to inform the Controller directly of such occurrences.

From: [email-masked]<mailto:[email-masked]> [mailto:[email-masked]] On Behalf Of [email-masked]<mailto:[email-masked]>
Sent: 05 September 2013 11:13
To: [email-masked]<mailto:[email-masked]>
Subject: [lmap] LMAP Framework issue #3 No negotiation between MA and Controller

Another issue:

Thoughts?
Thanks
phil

---

From: "Bugenhagen, Michael K" <[email-masked]>
Date: Thu, 12 Sep 2013 18:46:20 +0000

Phil,

   IMHO the MA's should be generating a "test results log" which is a short abbreviated log of tests to send to the Test Controller.
  It's NOT a wonderful thing to think all is well in the world only to find out at the end of the month that all your test results are filled with zeros.

So maybe a happy / sad indicator on the test scheduler would be a good thing

To do so

1)      MA would have to make a log of things like

a.       Test ID (should be unique)

b.      From point xxx

c.       To point yyy

d.      Test type zzz

e.      KPI result - pick anything that shouldn't be a zero

f.        Test closure condition - abort, normal, test refused, couldn't run test

This way we have information to figure out why schedule testing is or more importantly is not running as desired, what our failure rates are, and if we're collecting a bunch of zero's or not.

Cheers,
Mike

From: [email-masked] [mailto:[email-masked]] On Behalf Of [email-masked]
Sent: Thursday, September 12, 2013 10:45 AM
To: [email-masked]
Subject: [lmap] LMAP Framework issue #3 (take 2) Interactions between MA and Controller

Another issue:

Thoughts?
Thanks
phil

---

From: marcelo bagnulo braun <[email-masked]>
Date: Fri, 13 Sep 2013 11:46:57 +0200

Hi Phil,

Good stuff, thanks for doing this.

A couple of comments below... (hopefully nits that will give us some 
flexibility later on)

El 12/09/13 17:44, [email-masked] escribió:
>
> Thanks for the nice discussion, I’ve tried to re-write to reflect 
> where I think we’ve got to. I also changed the title to:
>
> contact a Controller. Defining a bootstrap mechanism is out of scope 
> of the LMAP WG.
>
> The basic interaction between a Controller and MA is that the 
> Controller sends an Instruction to the MA, detailing what Measurement 
> Tasks to do, when to do them, and how to report the Measurement 
> Results. In general we expect that the measurement system understands 
> what Measurement Methods the MA can do and therefore the Controller 
> can simply instruct the MA; there is no need for a negotiation 
> protocol (which would add complexity to the MA, Controller and Control 
> Protocol). However, it is possible that the MA is in fact not capable 
> of performing the requested Measurement Method, and so the Control 
> Protocol must allow the MA to reply with a suitable error code.
>

mmmm, I would rephrase the last sentence as follows:

The control protocol must allow the controller to learn that the MA was 
unable to perform the requested measurement method and some error code 
describing why.

I am suggesting this because as it is currently phrased, it seems that 
the control protocol needs to have a inmediate reply with the error, 
which is not what is covered in the information model, i think. In the 
information model, we have a logging and status information that can be 
queried where this information lies, but it is not returned as a 
response to an instruction.

> it is capable of. This could be useful:

I would rephrase as: "the controller can retrieve from the MA the list 
of measurement methods..."

the way it is currnelty written it seems the MA will spontaneously send 
the information to the controller

> [1] as part of the MA registering with the Controller. Note that in 
> some scenarios it is not needed, as the Controller will know the 
> information by other means, for example as part of the bootstrapping 
> process (using Tr-069 say) or because the MA is statically configured.
>
> ‘something goes wrong’ and the Controller forgets what the MA can do.
>
> MA can perform. It is not intended to indicate dynamic capabilities 
> like the MA’s currently unused CPU, memory or battery life.
>
> ** Comment: I’m not sure whether we reached consensus on this point.
>
> It is possible that later, when a MA is implementing the Instruction, 
> it does not perform a particular Measurement Task for some reason, 
> such as: the Measurement Peer is busy, or there is cross-traffic, … 
> The MA doesn’t inform the Controller, instead it is reported to the 
> Collector as part of the Measurement Report.
>

mmm, i would also mention that the MA may have a logging capability to 
store additional information about such events.

regards, marcelo

> The Collector may in turn tell the measurement system or even the 
> Controller, but this is out of scope of LMAP.
>
> ** Comment: are there any conditions when it’s critical for the 
> *From:[email-masked] [mailto:[email-masked]] *On Behalf 
> Of [email-masked]
> *Sent:* 05 September 2013 11:13
> *To:* [email-masked]
> *Subject:* [lmap] LMAP Framework issue #3 No negotiation between MA 
> and Controller
>
> Another issue:
>
> capabilities of its MAs (probably there are only one or a few classes 
> of MA), so negotiation about the MA’s capabilities would have little 
> benefit. It would also add complexity to the MA, Controller and 
> Control Protocol.
>
> It is possible that the MA can’t perform the requested Measurement 
> Task. So the Control Protocol needs to allow the MA to reply with an 
> error code. It has also been suggested that the Controller should be 
> able to ask the MA to send a list of all the Measurement Methods that 
> it is capable of, in case ‘something goes wrong’ and the Controller 
> forgets what the MA can do. Comment: this idea hasn’t had much 
> discussion yet, but appears in the Information Model i-d
>
> Thoughts?
>
> Thanks
>
> phil
>
>
>
> _______________________________________________
> lmap mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/lmap

---

From: <[email-masked]>
Date: Tue, 17 Sep 2013 10:58:55 +0100

Thanks Barbara.
thinking about your process question - we're now trying to write a 'protocol model' for the framework doc (rfc4101) - so the protocol will be described at this level.

Not sure I fully understood your point about Instruction.
I was trying to use it as defined in the terminology

<<Instruction: The description of Measurement Tasks to perform and the
   details of the Report to send.  The Instruction is sent by a
   Controller to a Measurement Agent.

The Instruction would be a message with

-       one of more Measurement Tasks

-       one of more Schedules

-       one or more Report Channels
I guess the first msg from Controller to MA includes all of these. Subsequent msgs might only update one of them, for instance the Schedule might get updated rather more often than the other two items.

From: STARK, BARBARA H [mailto:[email-masked]]
Sent: 12 September 2013 19:34
To: Eardley,PL,Philip,TUB8 R; [email-masked]
Subject: RE: [lmap] LMAP Framework issue #3 (take 2) Interactions between MA and Controller

Another issue:

Thoughts?
Thanks
phil

---

From: "STARK, BARBARA H" <[email-masked]>
Date: Tue, 17 Sep 2013 13:51:24 +0000

I see that your use of "Instruction" is consistent with its proposed LMAP terminology definition. And I have no problem with there being a term with that definition.

It would appear that my problem is with the choice of "Instruction" as the word to apply this definition to.

The proposed terminology definition uses "Instruction" in a manner that I don't consider completely consistent with the way I generally use the word.
In computing, "Instruction" is a very basic, low-level message. Such a message tells the computer to do something very specific. There are many, many instructions in a computer's "instruction set".
I've also seen this term used with other inter-device communication. As with computing "instructions", this sort of communication has many different types of messages, such that each desired action has its own "instruction".
In the absence of actually reading the lmap terminology definition, and forcing myself to accept that lmap is using this word in what I consider an industry-inconsistent way, I would misinterpret the use of this word (and did in fact misinterpret it).
For me, "instruction" in the context of some of the xml-based WAN configuration protocols we're looking at, would map to an RPC (Remote Procedure Call), if there were not a terminology definition telling me that I should not misinterpret the word in this manner.
I believe it is problematic to select a term that is so commonly used in the industry, and provide it with a definition that is not consistent with that common usage.

I believe the term could easily be replaced by:
   MA Configuration: The description of Measurement Tasks to perform and the
   details of the Report to send.  The MA Configuration is sent by a
   Controller to a Measurement Agent.

If we wanted to just call this Configuration, we could then add the sentence:
Also simply referred to as "Configuration" where the context is clear.

Barbara

From: [email-masked] [mailto:[email-masked]]
Sent: Tuesday, September 17, 2013 5:59 AM
To: STARK, BARBARA H; [email-masked]
Subject: RE: [lmap] LMAP Framework issue #3 (take 2) Interactions between MA and Controller

Thanks Barbara.

-       one of more Schedules

From: STARK, BARBARA H [mailto:[email-masked]]
Sent: 12 September 2013 19:34
To: Eardley,PL,Philip,TUB8 R; [email-masked]<mailto:[email-masked]>
Subject: RE: [lmap] LMAP Framework issue #3 (take 2) Interactions between MA and Controller

Another issue:

Thoughts?
Thanks
phil

---

From: "Bugenhagen, Michael K" <[email-masked]>
Date: Tue, 17 Sep 2013 14:00:45 +0000

Adding a bit to some of Barbara's input

What should be passed to a MA

1)      Schedule to initiate testing

2)      Policy of what testing to accept (who, what type, ....)

The big question I have is when does a MA check it's test schedule to see if can actually handle it.
I would expect that

a)      The MA do a cursory test capability check on it's test schedule to make sure it can do a test - IF NOT it should reject it (not a negotiation but a "accept / reject")

      Again this is to fix the "I thought everything was running ok" issue.

                      When Op's sets up a test to run overnight only to find out the bulk of the test points couldn't run it ....  May be the last time they use that system.

b)      The MA does a resource check to approve or deny a test once the time to test (schedule) is reached - I'm saying it should do test call admission pre-test-flight checks and then permit or deny the

      Test initiation.

From: [email-masked] [mailto:[email-masked]] On Behalf Of STARK, BARBARA H
Sent: Tuesday, September 17, 2013 8:51 AM
To: [email-masked]; [email-masked]
Subject: Re: [lmap] LMAP Framework issue #3 (take 2) Interactions between MA and Controller

Barbara

From: [email-masked]<mailto:[email-masked]> [mailto:[email-masked]]
Sent: Tuesday, September 17, 2013 5:59 AM
To: STARK, BARBARA H; [email-masked]<mailto:[email-masked]>
Subject: RE: [lmap] LMAP Framework issue #3 (take 2) Interactions between MA and Controller

Thanks Barbara.

-       one of more Schedules

Another issue:

Thoughts?
Thanks
phil

---

From: <[email-masked]>
Date: Tue, 17 Sep 2013 16:15:34 +0100

What do other people think?
I don't have a strong feeling either way.

From: STARK, BARBARA H [mailto:[email-masked]]
Sent: 17 September 2013 14:51
To: Eardley,PL,Philip,TUB8 R; [email-masked]
Subject: RE: [lmap] LMAP Framework issue #3 (take 2) Interactions between MA and Controller

Barbara

Thanks Barbara.

-       one of more Schedules

Another issue:

Thoughts?
Thanks
phil

---

From: "Carey, Timothy (Timothy)" <[email-masked]>
Date: Tue, 17 Sep 2013 15:21:36 +0000

Phil

If the definition of Instruction was limited to MA Configuration (which in the current definition it seems to be) then Barbara's definition is more clear.

BR,
Tim

From: [email-masked] [mailto:[email-masked]]
Sent: Tuesday, September 17, 2013 11:16 AM
To: [email-masked]; [email-masked]
Subject: Re: [lmap] LMAP Framework issue #3 (take 2) Interactions between MA and Controller

Barbara

Thanks Barbara.

-       one of more Schedules

Another issue:

Thoughts?
Thanks
phil

---

From: "Romascanu, Dan (Dan)" <[email-masked]>
Date: Tue, 17 Sep 2013 15:22:00 +0000

Neither do I (feel strongly either way). However I believe that Configuration risks to be also mis-leading, and not consistent with other usages of the term in the industry (or on the MA or the MA host). What about "Procedure" or "MA Procedure"?

Dan
(speaking as contributor)

From: [email-masked] [mailto:[email-masked]] On Behalf Of [email-masked]
Sent: Tuesday, September 17, 2013 5:16 PM
To: [email-masked]; [email-masked]
Subject: Re: [lmap] LMAP Framework issue #3 (take 2) Interactions between MA and Controller

Barbara

Thanks Barbara.

-       one of more Schedules

Another issue:

Thoughts?
Thanks
phil

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Tue, 17 Sep 2013 17:22:23 +0200

> Also simply referred to as "Configuration" where the context is clear.

MA Configuration seems too broad to me given the current definition of
Instruction:

   Instruction: The description of Measurement Tasks to perform and the
   details of the Report to send.  The Instruction is sent by a
   Controller to a Measurement Agent.

In other words, Instruction, as currently defined, is the subset of
the MA Configuration pertaining to Measurement Tasks and the Reports.
An unwieldy long term like 'Measurement Task and Report Configuration'
would be more precise but not very practical. So yes, Instruction may
not be ideal but so far I have not seen a good counter proposal.

/js

---

From: <[email-masked]>
Date: Tue, 17 Sep 2013 16:23:15 +0100

Thinking about this more there seem several cases:

1.    the MA cannot action the Instruction (for example, it doesn't include a parameter that is mandatory for the requested Measurement Method);

2.    the Measurement Task could not be executed (for example, the MA unexpectedly has no spare CPU cycles).

3.    the Measurement Task executes but (correctly) no test traffic is sent - a "pre-check test" - for example if the MA detects cross-traffic

discussing with trevor - it seems to make sense if 1 & 2 are reported by the MA to the Controller (don't want them polluting your measurement results), whilst 3 is reported by the MA to the Collector (as part of a normal Report).

Expanding on 1:
no value for a mandatory parameter
time of test is in past
type wrong, eg string given where expect integer
Schedule refers to a Measurement configuration or Report Channel that doesn't exist

Expanding on 2:
MA has crashed
MA doesn't (any longer) understand requested Method
MA has run out of CPU, memory, battery power
Collector has disappeared
MP has disappeared

From: Bugenhagen, Michael K [mailto:[email-masked]]
Sent: 17 September 2013 15:01
To: 'STARK, BARBARA H'; Eardley,PL,Philip,TUB8 R; [email-masked]
Subject: RE: [lmap] LMAP Framework issue #3 (take 2) Interactions between MA and Controller

      Test initiation.

Barbara

Thanks Barbara.

-       one of more Schedules

Another issue:

Thoughts?
Thanks
phil

---

From: "Sharam Hakimi" <[email-masked]>
Date: Tue, 17 Sep 2013 11:42:22 -0400

Would Command-List be acceptable. The definition would stay the same as
it is, just replace instruction with Command-List.

Command-List: The description of Measurement Tasks to perform and the
   details of the Report to send.  The Command-List is sent by a
   Controller to a Measurement Agent.

Sharam

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of
Juergen Schoenwaelder
Sent: Tuesday, September 17, 2013 11:22 AM
To: STARK, BARBARA H
Cc: [email-masked]; [email-masked]
Subject: Re: [lmap] LMAP Framework issue #3 (take 2) Interactions
between MA and Controller

On Tue, Sep 17, 2013 at 01:51:24PM +0000, STARK, BARBARA H wrote:
> I see that your use of "Instruction" is consistent with its proposed
LMAP terminology definition. And I have no problem with there being a
term with that definition.
> The proposed terminology definition uses "Instruction" in a manner
that I don't consider completely consistent with the way I generally use
the word.
> In computing, "Instruction" is a very basic, low-level message. Such a
message tells the computer to do something very specific. There are
many, many instructions in a computer's "instruction set".
> I've also seen this term used with other inter-device communication.
As with computing "instructions", this sort of communication has many
different types of messages, such that each desired action has its own
"instruction".
> In the absence of actually reading the lmap terminology definition,
and forcing myself to accept that lmap is using this word in what I
consider an industry-inconsistent way, I would misinterpret the use of
this word (and did in fact misinterpret it).
> For me, "instruction" in the context of some of the xml-based WAN
configuration protocols we're looking at, would map to an RPC (Remote
Procedure Call), if there were not a terminology definition telling me
that I should not misinterpret the word in this manner.
> I believe it is problematic to select a term that is so commonly used
in the industry, and provide it with a definition that is not consistent
with that common usage.

/js

---

From: "STARK, BARBARA H" <[email-masked]>
Date: Tue, 17 Sep 2013 15:41:57 +0000

<bhs> As it relates to a 'protocol model' for the framework, I'd want to be careful here. RFC 4101 is really about modelling of a brand new protocol that is going to be defined, or about providing a description of a particular protocol. In the context of LMAP, I thought we wanted to select existing protocols, and not design new ones. Therefore, it's not modelling from a protocol design perspective that we need to do, but identifying selection criteria for a protocol.  This is a subtle distinction. What we create should not attempt to describe "The LMAP Control Protocol" or "The LMAP Collector Protocol" (because hopefully there will not exist a protocol solely dedicated to these uses), but rather the criteria that a protocol would need to meet in order to be used for one of these purposes.

It's possible that multiple existing protocols will be able to meet the specified criteria. I recognize that it's a goal of this WG to then select one (for each purpose) and crown it as "The Protocol for IETF LMAP Control" or "The Protocol for IETF LMAP Data Collection". But it should also be recognized that the criteria alone may be useful, to entities who want to use a different protocol for the same purpose. I believe that the criteria should be written with that potential use in mind.

At a high level, I agree conceptually with RFC 4101 type of modelling. My real hope is that we not try to get into the level of detail that would be needed to model a brand new protocol, but limit it to the level of detail needed to select an existing protocol.
Barbara

---

From: "MORTON, ALFRED C (AL)" <[email-masked]>
Date: Tue, 17 Sep 2013 11:53:09 -0400

...
At a high level, I agree conceptually with RFC 4101 type of modelling. My real hope is that we not try to get into
the level of detail that would be needed to model a brand new protocol, but limit it to the level of detail needed
to select an existing protocol.
Barbara

Yes. I think what we intend, or what we had last night at least,
are very high-level models for various LMAP interactions.
Almost any protocol could fill the need, so your point is
well-taken.  Don't go too far at this stage...

Al

From: [email-masked] [mailto:[email-masked]] On Behalf Of STARK, BARBARA H
Sent: Tuesday, September 17, 2013 11:42 AM
To: [email-masked]; [email-masked]
Subject: Re: [lmap] LMAP Framework issue #3 (take 2) Interactions between MA and Controller

---

From: <[email-masked]>
Date: Tue, 17 Sep 2013 17:21:28 +0100

To give a bit of reasoning while I selected "Instruction" to start with...

-        There is also a configuration command/information about the general settings of the MA (such as MA ID, Group ID etc.). So configuration is taken (unless we change that as well)

-        It is specifically about getting the MA to do something - either conduct measurements and produce reports. The stuff in "Configuration" doesn't result in anything actually being done.

I think "instruction" is pretty descriptive and appropriate, but then I'm obviously biased.

Trevor.

Trevor Burbridge
Network Infrastructure & Innovation | BT Innovate & Design
Tel: 01473 645115
Fax: 01473 640929

This email contains BT information, which may be privileged or confidential. It's meant only for the individual(s) or entity named above. If you're not the intended recipient, note that disclosing, copying, distributing or using this information is prohibited. If you've received this email in error, please let me know immediately on the email address above. Thank you.
We monitor our email system, and may record your emails.
British Telecommunications plc
Registered office: 81 Newgate Street London EC1A 7AJ
Registered in England no: 1800000

From: [email-masked] [mailto:[email-masked]] On Behalf Of Romascanu, Dan (Dan)
Sent: 17 September 2013 16:22
To: Eardley,PL,Philip,TUB8 R; [email-masked]; [email-masked]
Subject: Re: [lmap] LMAP Framework issue #3 (take 2) Interactions between MA and Controller

Dan

From: [email-masked]<mailto:[email-masked]> [mailto:[email-masked]] On Behalf Of [email-masked]<mailto:[email-masked]>
Sent: Tuesday, September 17, 2013 5:16 PM
To: [email-masked]<mailto:[email-masked]>; [email-masked]<mailto:[email-masked]>
Subject: Re: [lmap] LMAP Framework issue #3 (take 2) Interactions between MA and Controller

Barbara

Thanks Barbara.

-        one of more Schedules

Another issue:

Thoughts?
Thanks
phil

---

From: "MORTON, ALFRED C (AL)" <[email-masked]>
Date: Tue, 17 Sep 2013 12:43:49 -0400

...

Trevor.

+1, and if there's still confusion, add the adjective "Measurement".

I've been using this term frequently in writing assignments,
and it hasn't tripped me up yet (which reminds me,
it's the term we used in the lmap charter, too).

Al

From: [email-masked] [mailto:[email-masked]] On Behalf Of [email-masked]
Sent: Tuesday, September 17, 2013 12:21 PM
To: [email-masked]; [email-masked]; STARK, BARBARA H; [email-masked]
Subject: Re: [lmap] LMAP Framework issue #3 (take 2) Interactions between MA and Controller

Trevor.

Trevor Burbridge

Dan

Barbara

Thanks Barbara.

-       one of more Schedules

Another issue:

Thoughts?
Thanks
phil

---

From: "Bugenhagen, Michael K" <[email-masked]>
Date: Tue, 17 Sep 2013 17:12:17 +0000

I'm hoping the group assumes Interoperability as a significantly high priority here.

   If we ever hope to have a DSL modem, or Cable modem, wireless phone, .... To be delivered with MA capabilities it really must have a standard command set or we'll end up with code variants that inhibit interop.

So I'm leaning to a common command set - I don't care what you call it, but it needs to be standard across the board.

From: [email-masked] [mailto:[email-masked]] On Behalf Of [email-masked]
Sent: Tuesday, September 17, 2013 11:21 AM
To: [email-masked]; [email-masked]; [email-masked]; [email-masked]
Subject: Re: [lmap] LMAP Framework issue #3 (take 2) Interactions between MA and Controller

Trevor.

Trevor Burbridge

From: [email-masked]<mailto:[email-masked]> [mailto:[email-masked]] On Behalf Of Romascanu, Dan (Dan)
Sent: 17 September 2013 16:22
To: Eardley,PL,Philip,TUB8 R; [email-masked]<mailto:[email-masked]>; [email-masked]<mailto:[email-masked]>
Subject: Re: [lmap] LMAP Framework issue #3 (take 2) Interactions between MA and Controller

Dan

Barbara

Thanks Barbara.

-       one of more Schedules

Another issue:

Thoughts?
Thanks
phil

---

From: "STARK, BARBARA H" <[email-masked]>
Date: Tue, 17 Sep 2013 19:04:24 +0000

Mike said:

<bhs> I disagree strongly. Certainly the Measurement Methods need to be totally interoperable -- but definition of those comes from ippm. That's not part of the "command set" we're talking about.

I believe very strongly that there is absolutely no requirement for the MAs procured by a DSL provider be controllable by the Controllers of a Cable operator. Nor that a provider's wireless devices be controllable by the same Controller as that provider's DSL modems. This, for me, was one of the big reasons for scoping this effort to MA and Controller belonging to the same domain/entity. Interoperability that needs to be guaranteed across entities (such as the Measurement Methods) requires the level of standardization you suggest. Interoperability internal to an entity can be defined and ensured by that entity.

The criteria regarding what makes for a good Control or Collector protocol must not include a requirement that all protocols used for these purposes have a common command set -- there is no justification I can think of to be that prescriptive. A protocol that meets these criteria should be expected to have a well-defined command set. But that really should go without saying -- that's obvious.

I fully expect that whatever IETF protocols LMAP chooses to recommend (that meet the criteria) will have a very specific and extremely well-defined command set. So any provider who wants to use that recommended protocol should be able to expect a certain degree of interoperability. Though, in my experience, if there is no certification program or test tools to ensure compliance with that protocol's specification, the provider may still need to put in a fair amount of effort and testing to ensure a satisfactory degree of interoperability between their procured MAs and Controller (assuming they come from different vendors -- single vendor selection tends to require less testing). The provider will probably take into account how knowledgeable and experienced their own test labs are with the protocol they select for use between their MAs and Controllers. Likewise, 3rd parties will make selections based on what makes sense for them.
Barbara

---

From: "Bugenhagen, Michael K" <[email-masked]>
Date: Tue, 17 Sep 2013 19:42:54 +0000

Barbara -

     I don't think we discussing the same inter-operability levels.

We have -

MA to MA
MC to MA
Stand alone MA in a home environment
Technology CPE (Service Provider or Customer Provided)

All of these have one single thing in common -  We want the test results to match.
  That cannot happen if we don't describe the testing sufficiently enough to ensure the "probe implementation" requirements are stateable, and testable.

Otherwise when a new test is constructed, we can't say that any one of the different technology devices can support running that test, and compare DSL, to Cable, to Satellite to LTE, .....
So if apples to apples results aren't a very high requirement I need to reset my thinking...

From: STARK, BARBARA H [mailto:[email-masked]]
Sent: Tuesday, September 17, 2013 2:04 PM
To: Bugenhagen, Michael K; [email-masked]'; [email-masked]; [email-masked]; [email-masked]
Subject: RE: [lmap] LMAP Framework issue #3 (take 2) Interactions between MA and Controller

Mike said:

---

From: "STARK, BARBARA H" <[email-masked]>
Date: Tue, 17 Sep 2013 22:10:37 +0000

Barbara -

We have -

MA to MA
MC to MA

<bhs> The command set (if I understand what you mean by that term) of a Controller to MA protocol has no impact on ensuring matching test results. Having a common information model that reflects a common set of well-defined Measurement Methods is what is needed to ensure matching test results. This is why we need to focus really hard on getting the Information Model well-defined. If that Information Model is accurately reflected in the data models of a dozen different protocols, defined by a dozen different SDOs or SIGs, and these protocols all meet a set of criteria for suitability to the task of being a good Control Protocol, then it doesn't matter which of these protocols an entity chooses to use between its Controller and its MAs.

Again, I understand that LMAP intends to recommend what this group believes to be the right IETF-defined protocol to use as a Control Protocol. I get that. Really I do. All I want is that RFCs don't get published that suggest this recommended protocol is the only possible suitable protocol. Therefore, specific command syntax is not appropriate in the framework as part of describing desired Control Protocol characteristics.
Barbara

---

From: marcelo bagnulo braun <[email-masked]>
Date: Wed, 18 Sep 2013 07:39:39 +0200

I guess i am biassed as well, but configuration, should be used for the 
initial configuration of the MA, while instruction seems like a specific 
thing that the MA is instructed to perform.

El 17/09/13 18:43, MORTON, ALFRED C (AL) escribió:
>
> …
>
> I think “instruction” is pretty descriptive and appropriate, but then 
> I’m obviously biased.
>
> Trevor.
>
> Of [email-masked]
> *Sent:* Tuesday, September 17, 2013 12:21 PM
> *To:* [email-masked]; [email-masked]; STARK, BARBARA H; 
> [email-masked]
> *Subject:* Re: [lmap] LMAP Framework issue #3 (take 2) Interactions 
> To give a bit of reasoning while I selected “Instruction” to start with…
>
> -There is also a configuration command/information about the general 
> settings of the MA (such as MA ID, Group ID etc.). So configuration is 
> taken (unless we change that as well)
>
> -It is specifically about getting the MA to do something – either 
> conduct measurements and produce reports. The stuff in “Configuration” 
> doesn’t result in anything actually being done.
>
> I think “instruction” is pretty descriptive and appropriate, but then 
> I’m obviously biased.
>
> Trevor.
>
> *Trevor Burbridge
> *
> This email contains BT information, which may be privileged or 
> confidential. It's meant only for the individual(s) or entity named 
> above. If you're not the intended recipient, note that disclosing, 
> copying, distributing or using this information is prohibited. If 
> you've received this email in error, please let me know immediately on 
> the email address above. Thank you.
> Of *Romascanu, Dan (Dan)
> *Sent:* 17 September 2013 16:22
> *To:* Eardley,PL,Philip,TUB8 R; [email-masked]; [email-masked]
> *Subject:* Re: [lmap] LMAP Framework issue #3 (take 2) Interactions 
> Neither do I (feel strongly either way). However I believe that 
> Configuration risks to be also mis-leading, and not consistent with 
> other usages of the term in the industry (or on the MA or the MA 
> host). What about “Procedure” or “MA Procedure”?
>
> Dan
>
> *From:[email-masked] <mailto:[email-masked]> 
> [mailto:[email-masked]] *On Behalf Of [email-masked] 
> <mailto:[email-masked]>
> *Sent:* Tuesday, September 17, 2013 5:16 PM
> *To:* [email-masked] <mailto:[email-masked]>; [email-masked] 
> <mailto:[email-masked]>
> *Subject:* Re: [lmap] LMAP Framework issue #3 (take 2) Interactions 
> I don’t have a strong feeling either way.
>
> *From:*STARK, BARBARA H [mailto:[email-masked]]
> *Sent:* 17 September 2013 14:51
> *To:* Eardley,PL,Philip,TUB8 R; [email-masked] <mailto:[email-masked]>
> *Subject:* RE: [lmap] LMAP Framework issue #3 (take 2) Interactions 
> I see that your use of “Instruction” is consistent with its proposed 
> It would appear that my problem is with the choice of “Instruction” as 
> the word to apply this definition to.
>
> The proposed terminology definition uses “Instruction” in a manner 
> that I don’t consider completely consistent with the way I generally 
> use the word.
>
> In computing, “Instruction” is a very basic, low-level message. Such a 
> many, many instructions in a computer’s “instruction set”.
>
> I’ve also seen this term used with other inter-device communication. 
> As with computing “instructions”, this sort of communication has many 
> “instruction”.
> For me, “instruction” in the context of some of the xml-based WAN 
> configuration protocols we’re looking at, would map to an RPC (Remote 
> in the industry, and provide it with a definition that is not 
> consistent with that common usage.
>
> Also simply referred to as “Configuration” where the context is clear.
>
> Barbara
>
> *From:[email-masked] <mailto:[email-masked]> 
> [mailto:[email-masked]]
> *Sent:* Tuesday, September 17, 2013 5:59 AM
> *To:* STARK, BARBARA H; [email-masked] <mailto:[email-masked]>
> *Subject:* RE: [lmap] LMAP Framework issue #3 (take 2) Interactions 
> thinking about your process question – we’re now trying to write a 
> ‘protocol model’ for the framework doc (rfc4101) – so the protocol 
> will be described at this level.
>
> -one of more Measurement Tasks
>
> -one of more Schedules
>
> -one or more Report Channels
>
> I guess the first msg from Controller to MA includes all of these. 
> Subsequent msgs might only update one of them, for instance the 
> Schedule might get updated rather more often than the other two items.
>
> *From:*STARK, BARBARA H [mailto:[email-masked]]
> *Sent:* 12 September 2013 19:34
> *To:* Eardley,PL,Philip,TUB8 R; [email-masked] <mailto:[email-masked]>
> *Subject:* RE: [lmap] LMAP Framework issue #3 (take 2) Interactions 
> How detailed is this Framework document supposed to be? Is it intended 
> to include the full list of criteria for a Control Protocol, or will 
> these criteria be documented separately? If the first, then I would 
> prefer for the criteria to be more precisely described, and easy to 
> identify as Control Protocol criteria/requirements. If the latter, 
> then I would prefer for the Framework to use more descriptive language 
> as to what is expected of the Control Protocol, rather than trying to 
> give names to all of the various expected functionality (e.g., 
> Instruction, registration). More detailed comments below.
>
> Barbara
>
> *From:[email-masked] <mailto:[email-masked]> 
> [mailto:[email-masked]] *On Behalf Of [email-masked] 
> <mailto:[email-masked]>
> *Sent:* Thursday, September 12, 2013 11:45 AM
> *To:* [email-masked] <mailto:[email-masked]>
> *Subject:* [lmap] LMAP Framework issue #3 (take 2) Interactions 
> <bhs> The use of the word “Instruction” here makes me uncomfortable, 
> because it implies (at least to me) a manner of communication that is 
> not quite consistent with the way most WAN-based 
> massive-number-of-controlled-devices control protocols work today. 
> “Instruction” suggests a manner of communication where the desired 
> action is a part of the basic message. Having separate message syntax 
> (actions) for everything the Controller wants the MA to do would 
> greatly increase the number of messages that need to go back and forth 
> between MA and controller. This sort of interaction is common in 
> protocols intended to work on LAN segments. It’s a very bad idea for 
> protocols intended to work over the WAN that are intended to scale to 
> large numbers of devices, and be highly extensible. I would prefer if 
> we did not attempt to give this function of the Controller to MA 
> interaction a proper name (denoted by a capital first letter). I would 
> also prefer if we described it as “the Controller configures the MA”, 
> using the word “configure” instead of “instruct”.
>
> I don’t think we’ve defined “negotiation protocol”. Probably because 
> we don’t need it. I would prefer if we got rid of that (negotiation 
> protocol) sentence and just used the last sentence as a positive 
> statement about what we do expect from the Control Protocol. For 
> example: The Control Protocol must support robust error reporting by 
> the MA, to provide the Controller with sufficiently-detailed reasons 
> for any failures in configuration.
>
> Is “measurement system” defined? I’m a bit unclear on what this is.
>
> Controller configures the MA, detailing what Measurement Tasks to do, 
> we expect that the Controller knows what Measurement Methods the MA 
> supports, such that the Controller can correctly configure the MA. 
> However, the Control Protocol must support robust error reporting by 
> the MA, to provide the Controller with sufficiently-detailed reasons 
> for any failures in configuration.
>
> <bhs> The word “register” also makes me uncomfortable, though not as 
> much as “Instruct”. This implies to me some very specific 
> functionality than I don’t consider completely necessary. But maybe if 
> “registration” were defined I would feel better about it. The 
> “**Comment” suggestion is too specific for the Framework, IMO. But if 
> Framework has detailed Control Protocol criteria, then I would reword 
> the statement as “The Control Protocol must allow the Controller to 
> request the MA supply the MA’s complete list of supported Measurement 
> Methods, in a concise format.” I would prefer if the above statements 
> were reworded as:
>
> [3] when requested by the Controller (e.g., if the Controller forgets 
> what the MA can do or otherwise wants to resynchronize what it knows 
> about the MA).
>
> <bhs> I would have no objection to including such capabilities as 
> optional elements of the information model. In fact, I think it would 
> be a good idea. But I don’t think this needs to be mentioned in the 
> Framework.
>
> Collector as part of the Measurement Report. The Collector may in turn 
> tell the measurement system or even the Controller, but this is out of 
> scope of LMAP.
>
> <bhs> Ditto my previous objection to “Instruction”. I believe the 
> information model needs to include status parameters that would allow 
> the Controller to request this info, at some level. This can be 
> optional to support for MA and Controller. I recommend rewording to:
>
> It is possible that an MA is unable to carry out a configured 
> Measurement Task. Possible reasons include the Measurement Peer being 
> busy, presence of cross-traffic, … The MA will report this information 
> to the Collector as part of the Measurement Report. The Collector may 
> in turn tell other systems (including, possibly, the Controller), but 
> this is out of scope of LMAP. The MA may also be able to inform the 
> Controller directly of such occurrences.
>
> *From:[email-masked] <mailto:[email-masked]> 
> [mailto:[email-masked]] *On Behalf Of [email-masked] 
> <mailto:[email-masked]>
> *To:* [email-masked] <mailto:[email-masked]>

---

From: marcelo bagnulo braun <[email-masked]>
Date: Wed, 18 Sep 2013 07:50:14 +0200

El 17/09/13 21:04, STARK, BARBARA H escribió:
>
> Mike said:
>
> I’m hoping the group assumes Interoperability as a significantly high 
> priority here.
>
> If we ever hope to have a DSL modem, or Cable modem, wireless phone, 
> …. To be delivered with MA capabilities it really must have a standard 
> command set or we’ll end up with code variants that inhibit interop.
>
> So I’m leaning to a common command set – I don’t care what you call 
> it, but it needs to be standard across the board.
>
> <bhs> I disagree strongly. Certainly the Measurement Methods need to 
> be totally interoperable -- but definition of those comes from ippm. 
> That’s not part of the “command set” we’re talking about.
>
> I believe very strongly that there is absolutely no requirement for 
> the MAs procured by a DSL provider be controllable by the Controllers 
> of a Cable operator. Nor that a provider’s wireless devices be 
> controllable by the same Controller as that provider’s DSL modems. 
> This, for me, was one of the big reasons for scoping this effort to MA 
> and Controller belonging to the same domain/entity. Interoperability 
> that needs to be guaranteed across entities (such as the Measurement 
> Methods) requires the level of standardization you suggest. 
> Interoperability internal to an entity can be defined and ensured by 
> that entity.
>
> The criteria regarding what makes for a good Control or Collector 
> protocol must not include a requirement that all protocols used for 
> these purposes have a common command set -- there is no justification 
> I can think of to be that prescriptive. A protocol that meets these 
> criteria should be expected to have a well-defined command set. But 
> that really should go without saying -- that’s obvious.
>

I think the key disagreement here is whether we believe lmap will 
produce one or more control protocols.
If we produce more than one protocol, i agree it doesnt make sense for a 
controller impelmenting protocol 1 to interoperate with a MA that 
implements protocol 2.

But if both implement the same protocol they shoudl have a common 
command set that they both understand. OTOH, the MA may not implement 
all the measurement methods that Controller wnats it to execute, but 
that is a different matter.

I guess the bottom line is that we may have different assumptions on the 
number of protocols lmap will define/reccomend, maybe?

Regards, marcelo

> I fully expect that whatever IETF protocols LMAP chooses to recommend 
> (that meet the criteria) will have a very specific and extremely 
> well-defined command set. So any provider who wants to use that 
> recommended protocol should be able to expect a certain degree of 
> interoperability. Though, in my experience, if there is no 
> certification program or test tools to ensure compliance with that 
> protocol’s specification, the provider may still need to put in a fair 
> amount of effort and testing to ensure a satisfactory degree of 
> interoperability between their procured MAs and Controller (assuming 
> they come from different vendors -- single vendor selection tends to 
> require less testing). The provider will probably take into account 
> how knowledgeable and experienced their own test labs are with the 
> protocol they select for use between their MAs and Controllers. 
> Likewise, 3^rd parties will make selections based on what makes sense 
> for them.
>
> Barbara
>
>
>

---

From: Brian Trammell <[email-masked]>
Date: Wed, 18 Sep 2013 08:21:03 +0200

hi Marcelo, all,

We should also consider that there's direct interoperability, and conceptual interoperability.

Direct interoperability: same wire protocol, same representation of the same information model. You plug two devices in, point them at each other or use the built-in discovery mechanism (if the protocol has one, which LMAP doesn't), and they Just Work, if we and the implementors have done their jobs right.

Conceptual interoperability: different wire protocols, different representations of two information models _largely derived_ from a common base information model. Here, devices using different protocols do not interoperate in terms of you plug them in and they talk to each other. But if you're really talking about different access technologies with completely different management stacks, interoperability in this sense doesn't make sense -- you can't even plug a cable modem into a DSL line, unless you're more creative (and more naïve) with physical layer adaptation than you should be. :)

But the common base information model (for the control protocol) here does have advantages. One, people who understand LMAP/DOCSIS build skills that transfer to LMAP/DSL or LMAP/LTE or what have you, because all the protocol data elements have the same names and the same meanings (and to the extent possible, the same encodings), access-technology-specific information elements notwithstanding. Conceptual interoperability also allows the easy design and implementation of post-LMAP control and analysis consoles that hide most of the specifics from the operator. 

And in case there really is a need for a controller to cross access technology boundaries (say, a cable network operator purchases a CLEC and decides to maintain the DSL network separately while unifying measurement), then conceptual interoperability allows lightweight protocol proxies to be built that translate control across that boundary. 

So you can have a common command set, and common control terminology, even if it's represented and transported differently, and get significant interoperability benefits even if you aren't directly interoperable. I'd say this approach is worth considering.

Best regards,

Brian

On Sep 18, 2013, at 7:50 AM, marcelo bagnulo braun <[email-masked]> wrote:

>> I’m hoping the group assumes Interoperability as a significantly high priority here.
>> 
>> If we ever hope to have a DSL modem, or Cable modem, wireless phone, …. To be delivered with MA capabilities it really must have a standard command set or we’ll end up with code variants that inhibit interop.
>> 
>> So I’m leaning to a common command set – I don’t care what you call it, but it needs to be standard across the board.
>> 
>> <bhs> I disagree strongly. Certainly the Measurement Methods need to be totally interoperable -- but definition of those comes from ippm. That’s not part of the “command set” we’re talking about.
>> 
>> I believe very strongly that there is absolutely no requirement for the MAs procured by a DSL provider be controllable by the Controllers of a Cable operator. Nor that a provider’s wireless devices be controllable by the same Controller as that provider’s DSL modems. This, for me, was one of the big reasons for scoping this effort to MA and Controller belonging to the same domain/entity. Interoperability that needs to be guaranteed across entities (such as the Measurement Methods) requires the level of standardization you suggest. Interoperability internal to an entity can be defined and ensured by that entity.
>> 
>> The criteria regarding what makes for a good Control or Collector protocol must not include a requirement that all protocols used for these purposes have a common command set -- there is no justification I can think of to be that prescriptive. A protocol that meets these criteria should be expected to have a well-defined command set. But that really should go without saying -- that’s obvious.
>> 
> 
> I think the key disagreement here is whether we believe lmap will produce one or more control protocols.
> If we produce more than one protocol, i agree it doesnt make sense for a controller impelmenting protocol 1 to interoperate with a MA that implements protocol 2.
> 
> But if both implement the same protocol they shoudl have a common command set that they both understand. OTOH, the MA may not implement all the measurement methods that Controller wnats it to execute, but that is a different matter.
> 
> I guess the bottom line is that we may have different assumptions on the number of protocols lmap will define/reccomend, maybe?
> 
> Regards, marcelo
> 
> 
> 
>> I fully expect that whatever IETF protocols LMAP chooses to recommend (that meet the criteria) will have a very specific and extremely well-defined command set. So any provider who wants to use that recommended protocol should be able to expect a certain degree of interoperability. Though, in my experience, if there is no certification program or test tools to ensure compliance with that protocol’s specification, the provider may still need to put in a fair amount of effort and testing to ensure a satisfactory degree of interoperability between their procured MAs and Controller (assuming they come from different vendors -- single vendor selection tends to require less testing). The provider will probably take into account how knowledgeable and experienced their own test labs are with the protocol they select for use between their MAs and Controllers. Likewise, 3^rd parties will make selections based on what makes sense for them.
>> 
>> Barbara
>> 
>> 
>>

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Wed, 18 Sep 2013 09:26:00 +0200

> Conceptual interoperability: different wire protocols, different representations of two information models _largely derived_ from a common base information model. Here, devices using different prot…

Frankly, for IP-layer and above measurements (this is what IPPM is all
about if I am not wrong), I fail to see why MAs would benefit from
being access technology specific. Obviously, MAs such as Ripe Atlas
probes or Sam Knows probes work on any access technology. The same
happens to be tru for pure software probes users run in Web browsers
or as apps. If we want to get LMAP MAs embedded into CPE devices, then
in at least some parts of the world, the CPE devices are owned by the
users and not the ISPs and they are pretty much access technology
independent (e.g. I have a cable model sitting in front of my CPE/NAT
router, for DSL lines you usually but a CPE/NAT router that does the
DSL stuff as well here in Germany but you can still use it behind the
Cable modem). I do see that there is a need to align the bootstrapping
to whatever the access technology typically does.

The IETF is usually about wire interoperability and the simple reason
is that an open standard allows to create a market around it which has
typically benefits for many stakeholders involved. I like to
understand why this does not apply to LMAP in general. In other words,
some of the use case should explain clearly why on the wire
interoperability is a non-achievable goal.

/js

---

From: "STARK, BARBARA H" <[email-masked]>
Date: Wed, 18 Sep 2013 13:32:33 +0000

> one or more control protocols.

 I am hoping lmap doesn't actually "produce" any protocols. But based on the charter and charter discussions, I'm fully expecting lmap to "select/extend" exactly one IETF-defined protocol as a Control protocol and exactly one as a Report protocol. What I want, is to make sure that lmap doesn't constrain its RFCs and recommendations in a manner that fails to recognize that other SDOs/SIGs will make other protocol recommendations, and entities will be free to choose which of these recommendations to use.

We have been discussing the framework draft and what is to be said about these 2 protocols in the framework draft. At the level of the framework draft, I would like to see text that describes the necessary and desirable functionality of a good Control/Report protocol. I am opposed to text that pre-supposes a particular Control or Report protocol, or to text that fails to recognize that other SDOs/SIGs will also make protocol recommendations. I would like the *framework* guidance in this area to be useful as input to all other orgs who want to select/extend a protocol for these purposes.

This does not prevent lmap from doing its own selection/extension of exactly one protocol for each purpose.
This is about having the framework draft not characterize its expectations of a Control or Report protocol in such a way as to ensure that only the IETF's selected/extended protocols can ever meet those expectations.
Barbara

---

From: marcelo bagnulo braun <[email-masked]>
Date: Wed, 18 Sep 2013 15:43:38 +0200

I think i agree with what you said.

---

From: "STARK, BARBARA H" <[email-masked]>
Date: Wed, 18 Sep 2013 14:22:20 +0000

> Frankly, for IP-layer and above measurements (this is what IPPM is all about
> if I am not wrong), I fail to see why MAs would benefit from being access
> technology specific. Obviously, MAs such as Ripe Atlas probes or Sam Knows
> probes work on any access technology. The same happens to be tru for pure
> software probes users run in Web browsers or as apps. If we want to get
> LMAP MAs embedded into CPE devices, then in at least some parts of the
> world, the CPE devices are owned by the users and not the ISPs and they are
> pretty much access technology independent (e.g. I have a cable model sitting
> in front of my CPE/NAT router, for DSL lines you usually but a CPE/NAT router
> that does the DSL stuff as well here in Germany but you can still use it behind
> the Cable modem). I do see that there is a need to align the bootstrapping to
> whatever the access technology typically does.
> 
> The IETF is usually about wire interoperability and the simple reason is that an
> open standard allows to create a market around it which has typically
> benefits for many stakeholders involved. I like to understand why this does
> not apply to LMAP in general. In other words, some of the use case should
> explain clearly why on the wire interoperability is a non-achievable goal.

I believe that it is important that lmap-defined MAs be able to operate usefully in the broader ecosystem of MAs that also perform Measurement Methods at physical and link layers, and other "custom" Measurement Methods.

The lmap framework must not restrict or assume that all Measurement Methods that an MA supports are defined by ippm, defined by IETF, or that they are necessarily intended for use by end users or regulators or will be provided or useful to any other entity (than the one in charge of the MA). If the framework is written in such a way as to restrict lmap-defined MAs to only ever be used for ippm metrics, then that's a pretty useless framework, IMO. Many operators will be unwilling to have both lmap MA and a separate layer 1/2 MA, on the same device, using separate Control and Report protocols. Certainly, the lmap framework must not assume that all MAs are operated by service providers. Likewise, it must not assume that they are not operated by service providers. The framework needs to be sufficiently flexible to allow for all of the use cases that are to be supported by it.

I have no desire for lmap to describe/define information or data models for these other Measurement Methods. I do have a desire to ensure that the lmap *framework* recognizes that MAs will include these, and to ensure that the *framework* recognizes this will occur and does not do anything to restrict or preclude this. 
Barbara

---

From: "Aamer Akhter (aakhter)" <[email-masked]>
Date: Thu, 19 Sep 2013 02:01:00 +0000

Inline with AA> 

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of STARK, BARBARA H
Sent: Wednesday, September 18, 2013 10:22 AM
To: Juergen Schoenwaelder; Brian Trammell
Cc: marcelo bagnulo braun; [email-masked]
Subject: Re: [lmap] LMAP Framework issue #3 (take 2) Interactions between MA and Controller

> Cable modem). I do see that there is a need to align the bootstrapping to whatever the access technology typically does.
> 
> some of the use case should explain clearly why on the wire interoperability is a non-achievable goal.

AA> I personally believe this is reasonable.

AA> Regarding metrics we've done similar things in IPFIX with the 'enterprise' range of IEs. Given that the IETF probably doesn't have much interest in defining RF channel reports in LMAP or the results of some DOCSIS test, it does seem to make sense to include options for other industry registries. Ie cablelabs etc. that would represent the lower layers. It does need to be organized though.

AA> Regarding the control/report protocol-- also agree. While we may have a recommended protocol  to start off with, the framework should be layered such that later on other protocols could be used instead. I do think of protocol Y has some compelling features (lower batter usage, is the standard management interface on cable modems, whatever) that usage should not be left undefined but should be brought back to the LMAP WG.  That said, I fully expect proprietary LMAP control/report mechanisms to be implemented.  Ffor example, for over-the-top content providers. In my mind this is ok- parts of the LMAP framework _should _be reusable. It's a good goal to have. 

Barbara

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Thu, 19 Sep 2013 09:06:53 +0200

On Wed, Sep 18, 2013 at 02:22:20PM +0000, STARK, BARBARA H wrote:
> I believe that it is important that lmap-defined MAs be able to
> operate usefully in the broader ecosystem of MAs that also perform
> Measurement Methods at physical and link layers, and other "custom"
> Measurement Methods.

We agree on this.

> The lmap framework must not restrict or assume that all Measurement
> Methods that an MA supports are defined by ippm, defined by IETF, or
> that they are necessarily intended for use by end users or
> regulators or will be provided or useful to any other entity (than
> the one in charge of the MA). If the framework is written in such a
> way as to restrict lmap-defined MAs to only ever be used for ippm
> metrics, then that's a pretty useless framework, IMO. Many operators
> will be unwilling to have both lmap MA and a separate layer 1/2 MA,
> on the same device, using separate Control and Report
> protocols. Certainly, the lmap framework must not assume that all
> MAs are operated by service providers. Likewise, it must not assume
> that they are not operated by service providers. The framework needs
> to be sufficiently flexible to allow for all of the use cases that
> are to be supported by it.

We even agree on this.

> I have no desire for lmap to describe/define information or data
> models for these other Measurement Methods. I do have a desire to
> ensure that the lmap *framework* recognizes that MAs will include
> these, and to ensure that the *framework* recognizes this will occur
> and does not do anything to restrict or preclude this.

Here is where we may likely draw different conclusions from this
paragraph. If you are having MAs in mind that do not today and will
never in the future talk IP, then the LMAP protocol likely won't apply
to them and there won't be on the wire interoperability. Is that what
you have in mind?

My point is that we need to scope the work and set priorities and I
believe achieving on the wire interoperability is an important goal
for IP enabled MAs and must be a major goal for this work to be
useful.

/js

---

From: "Bugenhagen, Michael K" <[email-masked]>
Date: Thu, 19 Sep 2013 15:53:18 +0000

Yes an Abstracted Test command framework / string is what I'm describing as a universal standard so it's repeatable... 
NOT a single protocol (this would be bad)

Defining a test is fine, however added information is required to ensure the test was executed in the same fashion so the test results are comparable.
- So we'll need to go beyond the test attributes in order to do this... (beyond frame size, ...)
 
Typical test implementation attributes I've had to define in the past

Test engine / Environmental factors
	-- Test engine (which TCP stack), no cross traffic, hour of day, target type or class, ....

Test procedure factors
	-- Stability factor used (delay, or trending and how much), thresholds for pass fail, Data trimming (95% of frames met x), ...

If these are not added to the Abstracted test command set I think we'll miss the target.

Regards,
Mike

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Juergen Schoenwaelder
Sent: Thursday, September 19, 2013 2:07 AM
To: STARK, BARBARA H
Cc: marcelo bagnulo braun; [email-masked]; Brian Trammell
Subject: Re: [lmap] LMAP Framework issue #3 (take 2) Interactions between MA and Controller

On Wed, Sep 18, 2013 at 02:22:20PM +0000, STARK, BARBARA H wrote:
> > Frankly, for IP-layer and above measurements (this is what IPPM is 
> > all about if I am not wrong), I fail to see why MAs would benefit 
> > from being access technology specific. Obviously, MAs such as Ripe 
> > Atlas probes or Sam Knows probes work on any access technology. The 
> > same happens to be tru for pure software probes users run in Web 
> > browsers or as apps. If we want to get LMAP MAs embedded into CPE 
> > devices, then in at least some parts of the world, the CPE devices 
> > are owned by the users and not the ISPs and they are pretty much 
> > access technology independent (e.g. I have a cable model sitting in 
> > front of my CPE/NAT router, for DSL lines you usually but a CPE/NAT 
> > router that does the DSL stuff as well here in Germany but you can 
> > still use it behind the Cable modem). I do see that there is a need to align the bootstrapping to whatever the access technology typically does.
> > 
> > The IETF is usually about wire interoperability and the simple 
> > reason is that an open standard allows to create a market around it 
> > which has typically benefits for many stakeholders involved. I like 
> > to understand why this does not apply to LMAP in general. In other 
> > words, some of the use case should explain clearly why on the wire interoperability is a non-achievable goal.
> 

We agree on this.

> that they are necessarily intended for use by end users or regulators 
> or will be provided or useful to any other entity (than the one in 
> charge of the MA). If the framework is written in such a way as to 
> restrict lmap-defined MAs to only ever be used for ippm metrics, then 
> that's a pretty useless framework, IMO. Many operators will be 
> unwilling to have both lmap MA and a separate layer 1/2 MA, on the 
> same device, using separate Control and Report protocols. Certainly, 
> the lmap framework must not assume that all MAs are operated by 
> service providers. Likewise, it must not assume that they are not 
> operated by service providers. The framework needs to be sufficiently 
> flexible to allow for all of the use cases that are to be supported by 
> it.

We even agree on this.

> and does not do anything to restrict or preclude this.

Here is where we may likely draw different conclusions from this paragraph. If you are having MAs in mind that do not today and will never in the future talk IP, then the LMAP protocol likely won't apply to them and there won't be on the wire interoperability. Is that what you have in mind?

My point is that we need to scope the work and set priorities and I believe achieving on the wire interoperability is an important goal for IP enabled MAs and must be a major goal for this work to be useful.

/js

---

From: "STARK, BARBARA H" <[email-masked]>
Date: Thu, 19 Sep 2013 17:39:43 +0000

> Here is where we may likely draw different conclusions from this paragraph.
> If you are having MAs in mind that do not today and will never in the future
> talk IP, then the LMAP protocol likely won't apply to them and there won't be
> on the wire interoperability. Is that what you have in mind?
> 
> My point is that we need to scope the work and set priorities and I believe
> achieving on the wire interoperability is an important goal for IP enabled MAs
> and must be a major goal for this work to be useful.

In the context of lmap, I am only interested in devices capable of communicating across the Internet. Devices that cannot talk IP are of no interest to me.
If the framework stated criteria such as "a Control protocol MUST operate over IP" and "a Reporting protocol MUST operate over IP", I would consider this very reasonable. But interoperability at the IP layer <> on-the-wire (application) interoperability.

In the content of Measurement Method protocols), on-the-wire interoperability for a given Measurement Method, among all endpoints claiming support for that Measurement Method is critical. This is something for ippm to work (as it relates to various IP Measurement Methods), and it is also something that the information model can help ensure (by including model elements for MAs to tell Controllers what Measurement Methods they support and including full information model support of the proposed ippm registry). The reason this is important, is that we understand IP Measurement Methods need to be able to run between MAs and MPs of different domains.

The lmap charter states "A key assumption constraining the initial work is that the measurement system is under the control of a single organization". The framework currently lists constraints of "Measurement system is under the direction of a single organization" and "Each MA may only have a single Controller at any point in time". These are very important constraints. These constraints mean to me that there is no need for a requirement for on-the-wire (application) interoperability of Control protocol or Reporting protocol between MAs/Controllers/Collectors of different organizations. Only intra-organization interoperability is required and each organization is free to determine what protocols to use in these contexts, without having any impact on inter-organization Measurement Method interoperability -- as long as the protocol properly supports the lmap information model.

<sarcasm> However, if there is consensus that the lmap framework has to mandate on-the-wire interoperability between *all* MAs, Controllers, and Collectors of different organizations, I would strongly recommend making use of RFC6919 language. Such as:
MAs and Controllers MUST use the lmap-selected Control Protocol (BUT WE KNOW YOU WON'T because organizations will have other protocols available for them to choose from, we recognize that these organizations will use criteria not listed in this document to determine which protocol is right for them, and we recognize that on-the-wire interoperability with other organizations' Controllers and MAs is likely not to be a criterion for many organizations).
Alternately, the framework could say "MAs and Controllers OUGHT TO use the lmap-selected Control Protocol". </sarcasm>
Personally, I would prefer for the framework to avoid any statement of this sort.

Some protocol criteria that might be useful are:
The Control Protocol MUST operate over IP. Note that where Controller and MA are in the same physical device, there is no need for a Control Protocol to be used.
The Control Protocol MUST include support for all Controller-MA elements of the Information Model. This includes support for all Measurement Methods identified in [ippm registry].
The Control Protocol MUST be extensible, so it can include support for Measurement Methods defined external to IETF, and for other industry-segment-specific or even organization-specific information elements.
Barbara

---

From: "STARK, BARBARA H" <[email-masked]>
Date: Thu, 19 Sep 2013 18:53:25 +0000

> Defining a test is fine, however added information is required to ensure the
> test was executed in the same fashion so the test results are comparable.
> 	-- Stability factor used (delay, or trending and how much), thresholds
> for pass fail, Data trimming (95% of frames met x), ...
> 

Hi Mike,
In http://tools.ietf.org/html/draft-burbridge-lmap-information-model-00 (which I hope will be used as a basis for lmap's Information Model deliverable), there's currently a section on Reporting Information that includes an "MA Context" element. IMO, the information you describe belongs here. Since you seem to have an idea of what you'd like to see included in Reporting Information, perhaps you could provide a more specific and "normalized" list that notes where you think the particular element is a string, datetime, enumerated list (and provide enumerations), etc. 

Perhaps the authors of that draft could say whether they'd like that sort of input, and if there's some sort of format they'd like to get that input in.

Anyway, I don't think there needs to be discussion about whether or not what you ask for is in scope of lmap. I think it's just a matter of getting the specifics of what it is fleshed out in the Information Model. At the level of the framework, I think what the framework currently says is great (see below). But maybe you would like to propose an additional bullet to expound in a single sentence or so a bit more on the idea of "context".
Barbara

------------------ from http://tools.ietf.org/html/draft-eardley-lmap-framework-02 ----
4.1.  Information Model
... snip ...
   The Information Model is divided into two main parts, each of which
   may be broken down into sub-parts:
... snip ...
   o  information about the Report: Information transmitted from the MA
      to the Collector including measurement results and the context in
      which they were conducted.  This includes:

      *  the MAs identifier, or perhaps a Group-ID to anonymise results

      *  the actual Measurement Results, including the time they were
         measured

      *  the details of the Measurement Task (to avoid the Collector
         having to ask the Controller for this information later)

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Thu, 19 Sep 2013 21:50:42 +0200

> The lmap charter states "A key assumption constraining the initial work is that the measurement system is under the control of a single organization". The framework currently lists constraints of "…

Frankly,

I think your logic is screwed. Yes, there is an assumption that the
measurement system is under the control of a single organization. You
seem to conclude from this that there is no value in defining a common
protocol. My understanding, however, is that this statement is there
since the stated assumption simplifies MAs and in particular the
protocol between MAs and controllers since there is no coordination
needed between multiple controllers from different domains. I believe
there is value in producing a common protocol. But like all protocols,
organizations will be free to decide whether they like to deploy them
or use something else. Hence, I do not really know what your issue is
here.

> Alternately, the framework could say "MAs and Controllers OUGHT TO use the lmap-selected Control Protocol". </sarcasm>

I will not comment on your attempt at producing sarcasm.

> The Control Protocol MUST include support for all Controller-MA elements of the Information Model. This includes support for all Measurement Methods identified in [ippm registry].

This is IMHO non-sense. Requiring support for _all_ measurements
methods registered is not meaningful. This MUST is not implementable
as the registry can change any time (nor is the implementation of all
measurement methods at a given snapshot in time necessarily
meaningful).

> The Control Protocol MUST be extensible, so it can include support for Measurement Methods defined external to IETF, and for other industry-segment-specific or even organization-specific informatio…

Whatever an 'information element' is here...

/js

---

From: "Carey, Timothy (Timothy)" <[email-masked]>
Date: Mon, 23 Sep 2013 01:21:30 +0000

Juergen and Barbara,

Sorry - I was trying to read this thread and I am not sure exactly what the issue/questions are, but there
Were a couple of topics that I wanted to express an opinion on.

A common framework (MA, Controllers, Collectors) are a good thing and we should continue this work.

We need to define the requirements of any protocol between the elements of the framework; with the goal that different organizations may want different protocol implementations. For example one organization may want to utilize the BBF CWMP and IPDR protocols for framework elements within their administrative domain. 

>From the requirements, the IETF could specify a protocol implementation but we must realize and allow for other protocols.

A common protocol neutral information model is a good thing; the LMAP group has a very good start in this process. However again we must allow for different implementation (representations) of the information model. Again an organization that would utilize the BBF CWMP and IPDR protocols would represent the information model as a CWMP data model. As such the Information model should be designed to allow for protocol specific attributes as first class entities and attributes. I can't see how that is done.

Likewise, the IETF could specify a protocol specific data model implementation but again we must realize and allow for other data model implementation of the information model.

As to the question of should framework elements be required to implement all the IPPM measurements; IMHO the answer to that is no unless the registry would contain only the basic of measurements. Instead we should ensure the framework and protocols allow for elements in the framework to state what the measurements and capabilities of a measurement they support. We should also allow for extensions to measurement registry; or have a mechanism for organizations to define their measurements that they would use in their administrative domain.

If we do these things, IMHO, we could have a framework that is used effectively by the different stakeholders.

Juergen, to answer your very real concern regarding inter-organization testing. If this was needed, wouldn't an organization that uses another protocol simply provide the controllers and collectors for these "unmanaged" MAs using the MA's protocol?

BR,
Tim

-----Original Message-----
From: Juergen Schoenwaelder [mailto:[email-masked]] 
Sent: Friday, September 20, 2013 2:51 AM
To: STARK, BARBARA H
Cc: [email-masked]
Subject: Re: [lmap] LMAP Framework issue #3 (take 2) Interactions between MA and Controller

Frankly,

/js

---

From: "STARK, BARBARA H" <[email-masked]>
Date: Mon, 23 Sep 2013 13:43:53 +0000

> > The Control Protocol MUST include support for all Controller-MA elements
> > of the Information Model. This includes support for all Measurement
> > Methods identified in [ippm registry].

> This is IMHO non-sense. Requiring support for _all_ measurements methods
> registered is not meaningful. This MUST is not implementable as the registry
> can change any time (nor is the implementation of all measurement methods
> at a given snapshot in time necessarily meaningful).

Interesting. When I design data models or databases and expect an important input (e.g., the ippm registry) to change relatively often, I use a "data-driven" design, rather than a "hard-coded" design. That is, I use pointers to the input, so that whenever the input changes, my data model or database can automatically change with it. The hard-coded approach would only provide for a mapping of the input up to the given snapshot in time. And not even including the full set of Measurement Methods listed in the registry at a given snapshot in time strikes me as undesirable, since that means the data model designers are overriding the recommendations of ippm as to what Measurement Methods (and their configuration and reporting information) need to be standardized. I thought there was agreement that ippm was to be trusted to create this registry and that lmap would simply take that as input, without further trying to restrict or edit the registry info. But perhaps I was mistaken. Personally, since my expertise lies in operations and not measurements, I will not be presuming to know better than the ippm people who create the registry. 

Well, in any case, this is a criterion I will use when making recommendations to my employer, and a design choice I will make in my external-to-IETF efforts. Hard-coded designs tend to be a little bit cheaper in the near-term, but much more expensive in the long-term. I've never seen people regretting data-driven designs that have been in the field for 5 or 10 or 20 years. I've often seen people regret hard-coding -- sometimes within a year.

But I have no problem with it not being reflected in any lmap document.
Barbara

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Mon, 23 Sep 2013 16:53:07 +0200

Barabara,

I never wrote about hard coding something. In fact, I mentioned that the
registry is subject to changes. I do not know where I lost you.

/js