Subject: [PMOL] FW: [ippm] Fwd: Liaison Statement from the Broadband Forum

From: Yaakov Stein <[email-masked]>
Date: Mon, 3 Sep 2012 14:33:07 +0000

Although I am sure that most of you have already seen this message,
I am forwarding it, because I believe that this liaison function should be the responsibility
of the performance directorate.

Y(J)S

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Reinhard Schrage
Sent: Sunday, September 02, 2012 22:28
To: [email-masked]; [email-masked]
Cc: [email-masked]; [email-masked]
Subject: Re: [ippm] Fwd: Liaison Statement from the Broadband Forum

I kind of like the idea of a liaison, as due to the work already done in the
IETF there is no need to re-invent already proven standards and procedures
and both sides may actually benefit.

Any actions already in place / planned?

Best regards
Reinhard Schrage
t:	+49 (0) 5137 909540
m:	+49 (0) 172  26.36.046
[email-masked]

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of
[email-masked]
Sent: 10 August 2012 08:34
To: [email-masked]
Cc: [email-masked]; [email-masked]
Subject: Re: [ippm] Fwd: Liaison Statement from the Broadband Forum

Matt,

I've read the liaision statement. The BBF informs us, that they plan to
"create a standard set of Broadband performance metrics to address on-going
regulatory, and consumer performance information needs."

BBF asks for guidance and support. From the document I couldn't identify the
architecture. I personally think IPPM members may volunteer to suggest IETF
metrics to BBF. But to me that requires that the architecture is clearly
defined (by BBF).

Regards,

Ruediger

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of
Matthew J Zekauskas
Sent: Friday, August 10, 2012 1:59 AM
To: IETF IPPM WG
Subject: [ippm] Fwd: Liaison Statement from the Broadband Forum

FYI, if folks here want to review.  Ron wants comments to [email-masked].

--Matt

-------------------

Folks,

The Broadband Forum has sent a liaison statement to the IETF regarding a
"New Project - Broadband Access Service Attributes and Performance
Measures".

Click on the following link for details:
https://datatracker.ietf.org/liaison/1179/.

Please review and send comments to this list.

--------------------------
Ron Bonica
vcard:       www.bonica.org/ron/ronbonica.vcf

---

From: "Romascanu, Dan (Dan)" <[email-masked]>
Date: Tue, 4 Sep 2012 09:01:19 +0200

It would be probably good if somebody can draft a response and provide
it as feedback as Ron asked initially. The response would probably point
to a number of RFCs that can help the Broadband Forum, probably most of
them resulting from work in IPPM. Based on this Ron and Benoit can sort
out with the Transport ADs who needs to answer the liaison letter. 

I also notice that the letter was also sent to ITU-T SG15Q14 and
SG12Q17. Al Morton is one of the leading contributors (also) in the
ITU-T work, so I suggest that we hear his input. 

Regards,

Dan

> From: [email-masked] [mailto:[email-masked]] On Behalf
Of
> Yaakov Stein
> Sent: Monday, September 03, 2012 5:33 PM
> To: [email-masked]
> Subject: [PMOL] FW: [ippm] Fwd: Liaison Statement from the Broadband
> Forum
> 
> Although I am sure that most of you have already seen this message, I
am
> forwarding it, because I believe that this liaison function should be
> the responsibility of the performance directorate.
> 
> Y(J)S
> 
> I kind of like the idea of a liaison, as due to the work already done
in
> the IETF there is no need to re-invent already proven standards and
> procedures and both sides may actually benefit.
> 
> "create a standard set of Broadband performance metrics to address on-
> going regulatory, and consumer performance information needs."
> 
> BBF asks for guidance and support. From the document I couldn't
identify
> the architecture. I personally think IPPM members may volunteer to
> suggest IETF metrics to BBF. But to me that requires that the
> architecture is clearly defined (by BBF).
> 
> Regards,
> 
> Ruediger
> 
> 
> _______________________________________________
> PMOL mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/pmol

---

From: Al Morton <[email-masked]>
Date: Tue, 04 Sep 2012 09:19:56 -0400

At 10:33 AM 9/3/2012, Yaakov Stein
wrote:

I am forwarding it, because I believe that this liaison function
should be the responsibility of the performance
directorate.

Interesting possibility.  I can certainly imagine members of

the Directorate commenting on this liaison, but why would we

claim responsibility for development of a reply?

IMO, the next steps are to agree on a test architecture and

divide the work of metric definition, methods of measurement,

and supporting protocol development among SDOs, where the

first step is to package existing standards that are relevant

and identify new standards development needed.

Some background on this Liaison:

The current text:

https://datatracker.ietf.org/liaison/1179/

is the first of two liaisons from BBF.  There is a more
recent

liaison from BBF's meeting last week which has not found its way

[email-masked] yet.  (I just forwarded it myself,
attached.)

The response should be to the combination of both liaisons (now that

there are two), not the first alone.
David Sinicrope

is IETF's Liaison Manager with BBF.

Apparently (second-hand info from a BBF attendee), the FCC has sent

a letter to both the BBF and the IETF requesting the development

of industry standards for broadband performance measurement,

and the FCC letter was discussed at BBF last week.

*Russ or someone in IETF/IAB should have received that letter.*

I don't have a copy.

Ron Bonica has asked for comments on the first liaison on ietf@ietf.

Some questions were raised among IPPM folks after Matt Z forwarded

Ron's request on ippm-list.

Finally, ITU-T SG 12 prepared an informative reply to the first

BBF liaison last June (also attached).

Al

---

From: Al Morton <[email-masked]>
Date: Tue, 04 Sep 2012 09:23:18 -0400

At 03:01 AM 9/4/2012, Romascanu, Dan (Dan)
wrote:

...I also notice that the letter was also sent to ITU-T SG15Q14 and

Sorry, should have read to the end of the thread before responding,

but at least I did as you asked...

Al

---

From: "Romascanu, Dan (Dan)" <[email-masked]>
Date: Wed, 5 Sep 2012 13:33:00 +0200

Hi Al,

I would let our (OPS) ADs decide together with the Transport ADs
sponsoring IPPM who is to be charged with drawing a response. 

One question - you wrote: 

*	IMO, the next steps are to agree on a test architecture and
divide the work of metric definition, methods of measurement, 
and supporting protocol development among SDOs

Do you believe that there is a need for new protocol development for BB
access performance? 

Regards,

Dan

		_____________________________________________
		From: [email-masked]
[mailto:[email-masked]] On Behalf Of Al Morton
		Sent: Tuesday, September 04, 2012 4:20 PM
		To: Yaakov Stein; [email-masked]
		Subject: MCAFEE E-MAIL SCAN ALERT!~RE: [PMOL] FW: [IPPM]
FWD: LIAISON STATEMENT FROM THE BROADBAND FORUM

		Attachment file :
bbf2012.1068.01.Liaison_to_IEEE__IETF__and_ITU_Regarding_WT-304_Status.d
oc\3.OLE
		Scanner Detected: Suspicious Extensions (Virus)
		Action taken : No Action Taken (File has a multiple
extension) 

		At 10:33 AM 9/3/2012, Yaakov Stein wrote:

				Although I am sure that most of you have
already seen this message, 
				I am forwarding it, because I believe
that this liaison function should be the responsibility of the
performance directorate.
		
			Interesting possibility.  I can certainly
imagine members of 
			the Directorate commenting on this liaison, but
why would we
			claim responsibility for development of a reply?
			
			IMO, the next steps are to agree on a test
architecture and
			divide the work of metric definition, methods of
measurement, 
			and supporting protocol development among SDOs,
where the 
			first step is to package existing standards that
are relevant 
			and identify new standards development needed.

			The current text:
			https://datatracker.ietf.org/liaison/1179/
			is the first of two liaisons from BBF.  There is
a more recent
			liaison from BBF's meeting last week which has
not found its way 
			[email-masked] yet.  (I just forwarded it
myself, attached.)
			The response should be to the combination of
both liaisons (now that 
			there are two), not the first alone.  David
Sinicrope <mailto:[email-masked]> 
			is IETF's Liaison Manager with BBF.
			
			Apparently (second-hand info from a BBF
attendee), the FCC has sent
			a letter to both the BBF and the IETF requesting
the development
			of industry standards for broadband performance
measurement,
			and the FCC letter was discussed at BBF last
week.
			*Russ or someone in IETF/IAB should have
received that letter.*
			I don't have a copy.
			
			Ron Bonica has asked for comments on the first
liaison on ietf@ietf.
			Some questions were raised among IPPM folks
after Matt Z forwarded
			Ron's request on ippm-list.
			
			Finally, ITU-T SG 12 prepared an informative
reply to the first 
			BBF liaison last June (also attached).
			
			Al

		 << File:
bbf2012.1068.01.Liaison_to_IEEE__IETF__and_ITU_Regarding_WT-304_Status.d
oc >>  << File: T09-SG12-120529-TD-GEN-0875!r1!!MSW-E.doc >>  << File:
ATT1826957.txt >>

---

From: Al Morton <[email-masked]>
Date: Wed, 05 Sep 2012 08:45:49 -0400

Hi Dan,

At 07:33 AM 9/5/2012, Romascanu, Dan (Dan) wrote:

One question
– you wrote:

Ø

Do you believe that there is a
need for new protocol
development for BB
access performance?

Not necessarily new protocols, but likely protocol extensions

to enable control among the various entities in the TBD
architecture.

For example, we have OWAMP and TWAMP test protocols, but they
probably

don't yet specify all the controls needed. Once the results are

measured, how are they transferred for further processing,

display, and archiving? What could we augment to do this
efficiently?

And, what SDO has the mandate to determine the testing
architecture?

Al

---

From: "Romascanu, Dan (Dan)" <[email-masked]>
Date: Wed, 5 Sep 2012 14:53:56 +0200

My understanding from the reading of their interaction with the IETF and ITU-T is that the BBF intent to take the responsibility of defining the testing architecture and the procedures, re-using components defined in other SDOs, and possibly pointing to existing gaps and requirements for extensions that could fill these. 

Is this your understanding also? Do you believe that we should do more than pointing to the protocols and metrics defined in the IETF and considering requirements for extensions if these show up? 

Regards,

Dan

From: Al Morton [mailto:[email-masked]] 
Sent: Wednesday, September 05, 2012 3:46 PM
To: Romascanu, Dan (Dan); Yaakov Stein; [email-masked]
Cc: Benoit Claise; Ronald Bonica
Subject: RE: [PMOL] FW: [IPPM] FWD: LIAISON STATEMENT FROM THE BROADBAND FORUM

Hi Dan,

Ø       IMO, the next steps are to agree on a test architecture and
divide the work of metric definition, methods of measurement,
and supporting protocol development among SDOs

Do you believe that there is a need for new protocol development for BB access performance? 

Not necessarily new protocols, but likely protocol extensions
to enable control among the various entities in the TBD architecture.
For example, we have OWAMP and TWAMP test protocols, but they probably
don't yet specify all the controls needed. Once the results are
measured, how are they transferred for further processing,
display, and archiving? What could we augment to do this efficiently?

And, what SDO has the mandate to determine the testing architecture?

Al

---

From: Al Morton <[email-masked]>
Date: Wed, 05 Sep 2012 10:05:52 -0400

I'd like to read the recent letter from the FCC, because I doubt

the roles and assignments of various SDOs are defined there.

BBF's claim to certain roles is understood at a high level.

If we agree it makes sense for BBF to define the architecture,

that's fine -- but let's agree on that first.

Al

At 08:53 AM 9/5/2012, Romascanu, Dan (Dan) wrote:

My understanding from the
reading of their interaction with the IETF and ITU-T is that the BBF
intent to take the responsibility of defining the testing architecture
and the procedures, re-using components defined in other SDOs, and
possibly pointing to existing gaps and requirements for extensions that
could fill these.

Is this your understanding also? Do you believe that we should do more
than pointing to the protocols and metrics defined in the IETF and
considering requirements for extensions if these show up?

Regards,

Dan

From: Al Morton
[
mailto:[email-masked]]

Subject: RE: [PMOL] FW: [IPPM] FWD: LIAISON STATEMENT FROM THE
BROADBAND FORUM

Hi Dan,

One question – you wrote:

Ø       IMO, the next steps are to agree on
a test architecture and

Al

---

From: Benoit Claise <[email-masked]>
Date: Thu, 20 Sep 2012 13:54:32 +0200

Hi,

[copying Wes Eddy, Transport A.D., responsible for IPPM]

This is what I can read in the liaison statement:

    This project will address the following topics.

    1.Specification of the test architecture

    2.A set of definitions to enable standard performance testing and
    reporting which includes:

    a.The Network Segment(s) being tested

    b.Tests and methodologies appropriate for each segment

    c.Test methods including test controller and test end-point requirements

    3.Recommended testing intervals for performance characterization and
    summarization purposes.

    4.Support of tests that span multiple operator networks.

    5.A standard method for quantifying and comparing access services.

    6.A common language for effective communication of service
    attributes between service providers and the customer.

    7.Templates for application providers on the network performance
    requirements.

    The BBF welcomes input and contributions on this project from
    partnering SDO's, and will keep you informed of progress on the
    Working Text.

My initial reaction, in no particular order:
- test architecture:
     active or passive?
     if active, does BBF plan on reusing TWAMP, OWAMP?
     if passive, does BBF plan on defining the architecture on its own?
- a set of definitions: it would be ideal to have performance metrics 
specified according to our BCP RFC 6390
- "support of tests than span multiple operator networks". Protocol 
extensions to what we have in the IETF (as mentioned by Al already)?
- basically, which IETF building blocks could BBF reuse? And which 
one(s) do they plan on reusing?
     Example: IPPM OWAMP and TWAMP, IPPM spatial and temporal 
aggregation, PMOL way of specifying metric, etc...
- we should separate what's generic (architecture and perf. metrics 
definition) from what's specific to an environment ("Recommended testing 
intervals for performance characterization and summarization purposes.", 
"A standard method for quantifying and comparing access services.", 
"Templates for application providers on the network performance 
requirements.")

Thoughts?

How should we draft the response?

Regards, Benoit.
>> My understanding from the reading of their interaction with the IETF 
>> and ITU-T is that the BBF intent to take the responsibility of 
>> defining the testing architecture and the procedures, re-using 
>> components defined in other SDOs, and possibly pointing to existing 
>> gaps and requirements for extensions that could fill these.
>>
>> Is this your understanding also? Do you believe that we should do 
>> more than pointing to the protocols and metrics defined in the IETF 
>> and considering requirements for extensions if these show up?
>>
>> Regards,
>>
>> Dan
>>
>>
>>
>> *From:* Al Morton [mailto:[email-masked] <mailto:[email-masked]>]
>> *Sent:* Wednesday, September 05, 2012 3:46 PM
>> *To:* Romascanu, Dan (Dan); Yaakov Stein; [email-masked]
>> *Cc:* Benoit Claise; Ronald Bonica
>> *Subject:* RE: [PMOL] FW: [IPPM] FWD: LIAISON STATEMENT FROM THE 
>> BROADBAND FORUM
>>
>> Hi Dan,
>>
>> One question -- you wrote:
>>

---

From: Al Morton <[email-masked]>
Date: Thu, 20 Sep 2012 09:11:12 -0400

Hi Benoit,

There's clearly a need to coordinate with BBF.

After some digging, we found that information attached

to a letter from the US FCC to the BBF proposes a

division of work between IETF and BBF, and that the

FCC intended to contact IETF formally with more details

(see the exchange I had with Ron below, it answers some

of your questions).

IETF has considerable products that are immediately applicable,

but there's additional work needed - some of which is critical

to the topic of broadband access measurement.

regards,

Al

At 11:31 AM 9/17/2012, Al Morton wrote:

Sounds good, lets get through
the 2544 stuff first...

At 11:01 AM 9/17/2012, Ronald Bonica wrote:

Al,

It took a while, but I just connected the dots….

Back in March, I gave a talk on IPv6 at the FCC. After the talk, Henning
Schulzrinne (CTO FCC) mentioned the measurement project and I offered
whatever help we (the IETF) could give. If you think that it would be
helpful, I would be glad to set up a conference call with you and
Henning.

Ron

From: Al Morton
[
mailto:[email-masked]]

Sent: Sunday, September 16, 2012 8:44 AM

To: Ronald Bonica

Subject: RE: WT-304 Liaison to IEEE, IETF, and ITU

At 10:04 AM 9/14/2012, Ronald Bonica wrote:

Would you be willing to draft a reply?

Ron

Yes, but we need to wait a bit, because IETF is intended

to receive a Liaison/letter from the FCC directly,

and we should respond to BBF after understanding

additional input from FCC.

The BBF mentions an "fcc document" they received
(attached)

in the 2nd BBF Liaison to IETF which arrived at end of August

(which is here

https://datatracker.ietf.org/liaison/1185/   )

It has been clarified that document was actually an attachment

to a short FCC letter, and the document came from the New America

Foundation (NAF), although it was clearly written with the FCC's

interest in mind.

This NAF document mentions informal and follow-up contacts with IETF

(near the bottom of page 2):

We have had informal discussion with the IETF regarding their possible
role in standardizing metrics used to characterize broadband performance
and intend to submit a request to the IETF to undertake a work program
towards this end. We believe that the data collection infrastructure
also requires standardization and are submitting this proposal to the
Broadband Forum to begin a work program to this end. It is our present
understanding that this proposal would match the objectives of the
Broadband Access Service Attributes and Performance Metrics (WT-304) work
program and would be appropriate for submission to this work group.

Since this describes a high-level division of work between BBF and
IETF,

I'll be looking for similar detail (as the document provides to BBF on
the

pages that follow) in the planned request to IETF.

Incidentally, this NAF document also refers to BBF's WT-304 doc as
though

it is a product with some substance, but I confirmed from a
co-editor

that it is just an outline and some text for the scope, nothing
more.

I've been in contact with our external affairs people who have
frequent

contact with the FCC - we're trying to determine the status of the

planned FCC contact.  I think that's worth waiting for, since it
should

portray the project as a joint effort and we can pursue it that way

in our reply to BBF.

Al

At 07:54 AM 9/20/2012, Benoit Claise wrote:

Hi,

1.         Specification of
the test architecture

2.         A set of
definitions to enable standard performance testing and reporting which
includes:

a.         The Network
Segment(s) being tested

b.         Tests and
methodologies appropriate for each segment

c.         Test methods
including test controller and test end-point requirements

3.         Recommended
testing intervals for performance characterization and summarization
purposes.

4.         Support of tests
that span multiple operator networks.

5.         A standard method
for quantifying and comparing access services.

6.         A common language
for effective communication of service attributes between service
providers and the customer.

7.         Templates for
application providers on the network performance requirements.

The BBF welcomes input and contributions on this project from
partnering SDO’s, and will keep you informed of progress on the Working
Text.

- test architecture:

active or passive?

if passive, does BBF plan on defining the architecture
on its own?

- "support of tests than span multiple operator
networks".  Protocol extensions to what we have in the IETF (as
mentioned by Al already)?

- basically, which IETF building blocks could BBF reuse? And which one(s)
do they plan on reusing?

Example: IPPM OWAMP and TWAMP, IPPM spatial and
temporal aggregation, PMOL way of specifying metric, etc...

- we should separate what's generic (architecture and perf. metrics
definition) from what's specific to an environment (" Recommended
testing intervals for performance characterization and summarization
purposes.", "A standard method for quantifying and comparing
access services.", "Templates for application providers on the
network performance requirements.")

Thoughts?

Regards, Benoit.

I'd like to read the recent
letter from the FCC, because I doubt

Al

Regards,

Dan

From: Al Morton [

mailto:[email-masked]]

To: Romascanu, Dan (Dan); Yaakov Stein;
[email-masked]

Hi Dan,

Al