Subject: [lmap] LMAP Framework issue #2 Admission control and measurement suppression

From: <[email-masked]>
Date: Tue, 3 Sep 2013 18:23:21 +0100

Another issue....

Admission control functionality pre-checks that it is a suitable moment to carry out a Measurement Task. It is the job of the Measurement Method to define such a pre-check - hence this is in the scope of IPPM rather than LMAP. Action could include:
*  the MA checking that there is no cross-traffic (ie that the user isn't already sending traffic) before the Measurement Task starts sending traffic;
*  the first part of the Measurement Task consisting of the MA asking the Measurement Peer whether it can send test traffic;
*  the first part of the Measurement Task consisting of traffic that probes the path to make sure it isn't overloaded;
*  the Measurement Task detecting the presence of cross-traffic after the MA (&/or MP) has started sending traffic.

Note that the LMAP WG does not define a generic admission control process, since the correct approach depends on the particular Measurement Task. This also means that the framework doesn't define a coordination process between MAs; whilst a measurement system may define coordinated Measurement Schedules across its various MAs, there is no direct coordination between MAs.

In addition the Controller may want to send a "suppress" message to some MAs so that they temporarily stop performing Active Measurement Tasks - perhaps there is some unexpected network issue and the ISP wants to eliminate inessential traffic. Since some Measurement Tasks involve the MP sending traffic to the MA, so this suggests the MA may sometimes need to send a "suppress" message to the MP.

Discussion needed...
1. the Broadband Forum liaison mentioned a generic admission process, but my impression is that people no longer think this is needed. Just wanted to make sure I got that right.
2. I've heard a couple of comments that suppression is needed, but we haven't had much discussion - do people agree?
3. the Information Model should discuss how suppression gets recorded in the results (perhaps the on-going results get scrapped - perhaps the results are marked with a warning - perhaps it depends on the metric?)
4. suppress is a bit messy if the MP is sending traffic to the MA (eg download speed test). One MP may be sending to several MAs. perhaps the suppress msg should go directly to the MP? Of course if the MP doesn't have lmap functionality then there's nothing that can be done in any case.

Comments?
Thanks
phil

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Tue, 3 Sep 2013 21:38:30 +0200

> Note that the LMAP WG does not define a generic admission control process, since the correct approach depends on the particular Measurement Task. This also means that the framework doesn't define a…

I am wondering whether 'admission control' is the right/best
term. Ideally, for certain longer running tests, one should not just
check before the test whether certain constraints (e.g. no cross
traffic) are satisfied but continue checking also during test
execution.
 
> 1. the Broadband Forum liaison mentioned a generic admission process, but my impression is that people no longer think this is needed. Just wanted to make sure I got that right.

I tend to assume that the constraints that need to be satisfied before
and during test execution are highly test dependent. Hence, I am not
sure a 'generic admission process' is that useful.

> 2. I've heard a couple of comments that suppression is needed, but we haven't had much discussion - do people agree?

Having a button to quickly and temporarily turn things off is often
very useful. I think it is not essential to get started but surely a
nice to have feature (could be an LMAP 2.0 feature if it is too hard
to do in 1.0).

> 3. the Information Model should discuss how suppression gets recorded in the results (perhaps the on-going results get scrapped - perhaps the results are marked with a warning - perhaps it depends…

Supression to me means tests are not executed and hence there are no
results. The logging mechanism should in my view log the beginning/end
of suppression.

> 4. suppress is a bit messy if the MP is sending traffic to the MA (eg download speed test). One MP may be sending to several MAs. perhaps the suppress msg should go directly to the MP? Of course if…

I think the LMAP model is based on the assumption that the controller
talks to the MA and the MA initiates and controls the tests. Hence, an
MP should not blindly send test traffic to MAs I think.

/js

-- 
Juergen Schoenwaelder           Jacobs University Bremen gGmbH
Phone: +49 421 200 3587         Campus Ring 1, 28759 Bremen, Germany
Fax:   +49 421 200 3103         <http://www.jacobs-university.de/>

---

From: Paul Aitken <[email-masked]>
Date: Tue, 03 Sep 2013 22:27:30 +0100

Phil,

> Another issue....
>
> Admission control functionality pre-checks that it is a suitable 
> moment to carry out a Measurement Task. It is the job of the 
> Measurement Method to define such a pre-check -- hence this is in the 
> scope of IPPM rather than LMAP. Action could include:
>
> *  the MA checking that there is no cross-traffic (ie that the user 
> isn't already sending traffic) before the Measurement Task starts 
> sending traffic;
>
> *  the first part of the Measurement Task consisting of the MA asking 
> the Measurement Peer whether it can send test traffic;
>
> *  the first part of the Measurement Task consisting of traffic that 
> probes the path to make sure it isn't overloaded;
>
> *  the Measurement Task detecting the presence of cross-traffic after 
> the MA (&/or MP) has started sending traffic.
>

If there are multiple independent MAs in a single device (or, in close 
proximity), a race condition could easily occur:

Suppose two MAs require no cross traffic for their tests - so both MAs 
wait for suitably quiescent conditions. Then they both query their MPs 
to obtain test traffic - at which point each MA sees the other's traffic 
and aborts. Rinse, repeat...

Or, once several MAs have obtained test traffic, the volume causes a CPU 
or interface overload which makes all the MAs abort.

So a certain amount of cooperation may be required between multiple MAs. 
This may be both necessary and out of scope...

> Note that the LMAP WG does not define a generic admission control 
> process, since the correct approach depends on the particular 
> Measurement Task. This also means that the framework doesn't define a 
> coordination process between MAs; whilst a measurement system may 
> define coordinated Measurement Schedules across its various MAs, there 
> is no direct coordination between MAs.
>
> In addition the Controller may want to send a "suppress" message to 
> some MAs so that they temporarily stop performing Active Measurement 
> Tasks -- perhaps there is some unexpected network issue and the ISP 
> wants to eliminate inessential traffic. Since some Measurement Tasks 
> involve the MP sending traffic to the MA, so this suggests the MA may 
> sometimes need to send a "suppress" message to the MP.
>

Definitely. However, the controller should always have an overall 
picture of what's going on in the network and be co-ordinating activity, 
so suppression is rarely required.

> Discussion needed...
>
> 1. the Broadband Forum liaison mentioned a generic admission process, 
> but my impression is that people no longer think this is needed. Just 
> wanted to make sure I got that right.
>
> 2. I've heard a couple of comments that suppression is needed, but we 
> haven't had much discussion -- do people agree?
>

I believe that it may be useful and should be designed for.

> 3. the Information Model should discuss how suppression gets recorded 
> in the results (perhaps the on-going results get scrapped -- perhaps 
> the results are marked with a warning -- perhaps it depends on the 
> metric?)
>

The MA should abort the test, append a "suppressed" message, and output 
the results that it has. Potentially the suppression message should have 
a "and don't report" option.

> 4. suppress is a bit messy if the MP is sending traffic to the MA (eg 
> download speed test). One MP may be sending to several MAs. perhaps 
> the suppress msg should go directly to the MP? Of course if the MP 
> doesn't have lmap functionality then there's nothing that can be done 
> in any case.
>

The controller might not know which MPs are in use. eg, the test might 
leave MP selection up to the MA. eg, ping the local DNS server 100 times.

P.

---

From: Paul Aitken <[email-masked]>
Date: Tue, 03 Sep 2013 22:29:15 +0100

Juergen,

> nice to have feature (could be an LMAP 2.0 feature if it is too hard

+1

> results. The logging mechanism should in my view log the beginning/end

Perhaps there are two requirements: "stop+report" versus "suppress" 
(with no report) ?

> MP should not blindly send test traffic to MAs I think.

+1.

The controller shouldn't talk to the MP / MPs.

P.

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Wed, 4 Sep 2013 09:49:22 +0200

On Tue, Sep 03, 2013 at 10:27:30PM +0100, Paul Aitken wrote:
> 
> If there are multiple independent MAs in a single device (or, in
> close proximity), a race condition could easily occur:
> 
> Suppose two MAs require no cross traffic for their tests - so both
> MAs wait for suitably quiescent conditions. Then they both query
> their MPs to obtain test traffic - at which point each MA sees the
> other's traffic and aborts. Rinse, repeat...
> 
> Or, once several MAs have obtained test traffic, the volume causes a
> CPU or interface overload which makes all the MAs abort.
> 
> So a certain amount of cooperation may be required between multiple
> MAs. This may be both necessary and out of scope...
> 

There will be many other things that can interfere with the tests
executed by an MA - not just other MAs. To make the system robust, MAs
and more specifically the tests need to carefully check as much as is
reasonable that the measurements are not impacted by lack of local
resources or network resources.

In other words, putting effort on coordination between multiple MAs
while lots of other stuff can still interfere with measurements seems
somewhat like a waste of time to me. If a test is sensitive to local
cross traffic, measure cross traffic and discard or label results that
may be wrong. If a test requires sufficient CPU cycles, check the
system load or the amount of time measurement processes are blocked
waiting for the CPU. If a test is sensitive to network disruptions,
check that the link status and network configuration did not change
during test execution. And in general, collect sufficient samples such
that any sufficient statistical evidence can be produced even if there
are occasional measurement errors an even well written test won't be
able to detect.

My point is that spending effort to coordinate multiple MAs while
everything else happening around an MA remains uncoordinated and
uncontrolled may not be an efficient use of resources.

/js

---

From: marcelo bagnulo braun <[email-masked]>
Date: Wed, 04 Sep 2013 10:52:27 +0200

Hi,

comments below...

El 03/09/13 19:23, [email-masked] escribió:
> Another issue….
> Measurement Method to define such a pre-check – hence this is in the 
> isn’t already sending traffic) before the Measurement Task starts 
> sending traffic;
>
> probes the path to make sure it isn’t overloaded;
>
> Measurement Task. This also means that the framework doesn’t define a 

My take is that this is test/measurement specific and should be taken 
care of in IPPM when each metric is defined.

> In addition the Controller may want to send a “suppress” message to 
> Tasks – perhaps there is some unexpected network issue and the ISP 
> sometimes need to send a “suppress” message to the MP.
>

I agree we need this.

We had a discussion about this a while ago and one specific issue that i 
found very useful is to distinguish between measurements that are 
running and measurements that are schedulled to run.

Suppose you have schedulled to run 10 TCP throughput tests that last 5 
seconds each and they are to be executed every 5 minutes.

Suppose that at time T1 the MA is running the first of the TCP throuput 
tests.

Suppose that at time T1 we send the suppress message: what are expecting 
to suppress?
a- abort the ongoing test and suppress the next 9 tests, or,
b- finnish the ongoing test and suppress the next 9 tests

I think b is easy, b not so much.

> Discussion needed…

While i see Paul's argumet for needing this, i would expect that in 
general metrics will have a random delay to start that will solve some 
of these issues.

So, i think i dont think i see too much value in a generic mechanism

> 2. I’ve heard a couple of comments that suppression is needed, but we 
> haven’t had much discussion – do people agree?
>

agree we need this, please see the discussion above on the timescales, 
to nail down exactly what we need.

> in the results (perhaps the on-going results get scrapped – perhaps 
> the results are marked with a warning – perhaps it depends on the metric?)
>

I think this fits into the logging

> doesn’t have lmap functionality then there’s nothing that can be done 
> in any case.
>

mmm, i think the MP should not talk to the controller. It is cleaner 
this way. Usually, the MA should be able to send an cancel message to 
the MP (like a TCP RST) and abort the measurement, no?

Regards, marcelo

> Comments?
>
> Thanks
>
> phil
>
>
>
> _______________________________________________
> lmap mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/lmap

---

From: <[email-masked]>
Date: Wed, 4 Sep 2013 10:50:11 +0100

> > Tasks - perhaps there is some unexpected network issue and the ISP
> We had a discussion about this a while ago and one specific issue that
> i found very useful is to distinguish between measurements that are
> Suppose that at time T1 we send the suppress message: what are
> expecting to suppress?

[phil] this is a very good point.
Assuming you mean "b is easy, a not so much" - I agree 
Would it be a reasonable target is for suppression to solve b, but in general not a (perhaps it would be possible for some tests, but not a general capability)?

---

From: "MORTON, ALFRED C (AL)" <[email-masked]>
Date: Tue, 10 Sep 2013 13:46:50 -0400

Again late to the party, comments below:

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On Behalf Of
> [email-masked]
> Sent: Wednesday, September 04, 2013 5:50 AM
> To: [email-masked]; [email-masked]
> Subject: Re: [lmap] LMAP Framework issue #2 Admission control and
> measurement suppression
> 
> Would it be a reasonable target is for suppression to solve b, but in
> general not a (perhaps it would be possible for some tests, but not a
> general capability)?
> 

Like everyone else, I think we need suppression, with or without report.

Most MA - MP test protocols we might use allow some way for the MA to abort the 
measurement in-progress. The only case I can think-up where abort is difficult
is where the MP is sending an un-acknowledged packet stream, and there is no
test control protocol available for the MA to say "STOP".

In TWAMP, the Stop control or the MA suspending transmission will eventually
starve the MP in a "responder" role (operating normally in "send-on-receive"),
and starving the MP would work in other protocols with a responder, too.

With a TCP download from MP, the MA could send Zero-Window or ditch the connection.

So, I think we might solve *almost* all the abort cases by telling the MA and 
relying on the measurement protocol features or mandatory Transport Protocol
feedback. Abort cannot be mandatory, but could be Recommended.

Al

---

From: marcelo bagnulo braun <[email-masked]>
Date: Wed, 11 Sep 2013 08:32:17 +0200

> feedback. Abort cannot be mandatory, but could be Recommended.

right, but this seems specific to the test the MA is performing, so it 
is unclear to me if we can build this into the lmap protocol as a 
generic feature, no?

---

From: "MORTON, ALFRED C (AL)" <[email-masked]>
Date: Wed, 11 Sep 2013 07:28:58 -0400

> From: marcelo bagnulo braun [mailto:[email-masked]]
...
> Al wrote:
Hi Marcelo,

I must be missing some detail - 

if we can build an effective suppression message into LMAP Control protocol
to cover cases "a" and "b", knowing that "a" will not always be possible:
>>> b- finish the ongoing test and suppress the next 9 tests

and the MA can receive an unexpected LMAP Control message 
and take immediate action on the measurement function 
(whether passive or active, OWAMP, TWAMP, IPSLA, TCP, ping, other) 
then it seems there are enough pieces to make suppression/abort work.

what am I missing? 
Al

---

From: marcelo bagnulo braun <[email-masked]>
Date: Thu, 12 Sep 2013 09:00:44 +0200

> then it seems there are enough pieces to make suppression/abort work.

right, the thing is that the actual effect of the suppression message 
would be dependent on the test being executed, as you say. I am not sure 
this is a good idea. Maybe it is the best we can do. Not sure at this point.

Regards, marcelo