Subject: Re: [PMOL] Proposed Adoption of draft-malas-performance-metrics-08
From: Cullen Jennings <[email-masked]>
To: [email-masked]
Date: Thu, 31 Jan 2008 20:36:39 -0800
List-Id: Performance Metrics at Other Layers <pmol.ietf.org>

Hi,

I'd like to make a few comments - some in my role as one of the RAI  
ADs and some just as an individual contributor.

With my AD hat on,

This draft was bounced around several places before we figured out  a  
good place to get this work done. I worked with Dan in his OPs AD role  
and with the chairs of the related WGs to try and determine where we  
wanted this work to happen.  We came to the conclusion that this was  
the best WG to do the work in. Now that does not mean the IETF has to  
do this work at all, but if we are going to do it, I think this is the  
right place to do it. I know that multiple people in the RAI area  
expressed their opinions that this work was worth doing. Of course I  
want the work to meet the quality level expected by this WG - that was  
part of the reasons Dan and I believed this group was the best place  
to do the work. I would expect that would mean evolving this draft to  
be consistent with the framework document as they both get closer to  
WGLC. This evolution certainly could be done after the draft was  
adopted as a WG document. I don't think the WG has to finish the  
framework document before they can adopt this and start doing work -  
both can happen in parallel if the WG choices to do that. The  
important thing to me is that when this WG sends the drafts to IESG,  
the WG has consensus that they are of adequate quality and consistency  
that they should be published as RFCs.

I note that we IETF Last Called the charter text that has
   2. A proof-of-concept RFC defining performance metrics for SIP, based
      on draft-malas-performance-metrics. This memo would serve as an
      example of the framework and the PMOL development process in the  
IETF.
and I don't believe I received any comments about this not being a  
good starting point.

 From a RAI point of view, I don't claim to know what is the best  
starting document to get this work done but the RAI folks would like  
to see some of the issues this draft addresses described in some RFC.

WIth my individual contributor hat on,

Of all the proposed drafts that this WG could adopted to meet it  
milestones, this one seems to be the best :-) I support adopting it as  
a WG item. I'm sure I don't know what needs to change to make it  
"right" but once it is a WG draft instead of an individual draft,  
whatever the consensus of the WG is can get put into this draft. I  
think this will be the fastest path to finishing the work and having  
people agree on it.

Thanks, Cullen

On Jan 23, 2008, at 7:28 AM, Al Morton wrote:

> PMOL Working Group:
>
> A comment period for the Internet-Draft on
>
> "SIP End-to-End Performance Metrics"
>
> <draft-malas-performance-metrics-08.txt>
>
> will be open from 23 January 2008 through 6 February 2003.
>
> A URL for this Internet-Draft is:
> http://tools.ietf.org/html/draft-malas-performance-metrics-08
>
> The purpose of this comment period is to examine whether there is  
> PMOL WG
> consensus to adopt this draft as the basis for the SIP Performance  
> Metrics
> "Proof of Concept" work item described in the PMOL WG Charter,
> http://www.ietf.org/html.charters/pmol-charter.html
> and if so to progress this as a Working Group Draft.
>
>
> STATUS
>
> When this topic was raised at the IETF-70 PMOL session, there
> was substantial readership and some support expressed to take-up
> this version as the WG draft. There were also some
> reservations expressed at the meeting about how well
> the current draft matches the (evolving and not yet complete)
> framework draft. Thus, completing this milestone will require
> further development of the framework (and process) draft, as well.
>
> Also, this draft has enjoyed interest and review of the SIPPING WG,
> and members of the SIP development community continue to apply
> their expertise to fine-tune the performance metric definitions.
>
> Please weigh-in with any comments to this list or to the co-chairs:
> <[email-masked]> and <[email-masked]>.
>
> Al Morton
> Alan Clark
> pmol co-chairs
>
> Note: The current version of the framework and process draft is here:
> http://tools.ietf.org/html/draft-morton-perf-metrics-framework-01
>
>
>
>
> _______________________________________________
> PMOL mailing list
> [email-masked]
> https://www1.ietf.org/mailman/listinfo/pmol

_______________________________________________
PMOL mailing list
[email-masked]
http://www.ietf.org/mailman/listinfo/pmol

Date: Thu, 31 Jan 2008 10:23:00 -0500
To: "Daryl Malas" <[email-masked]>, [email-masked]
From: Al Morton <[email-masked]>
Subject: Re: [PMOL] SIP performance metrics draft
Cc: 

Hi Daryl,

A few more points:

At 07:54 PM 1/29/2008, Al Morton wrote:
>Clock Accuracy and Error:  Since most of the SIP metrics are time
>intervals measured at a single UA, time sync between two remote
>points is not an issue. Time interval accuracy for the single
>clock should be identified as a source of error, however.

It will help to designate one of the two time stamps (TB or TS)
as "the" time each individual test was conducted.  Clearly,
TB is the most suitable for this use. TB is the time that is
controled in an Active test scenario. I suggest adding this
to the current definition of TB:

... TB represents the time at which each request-response test begins,
and SHALL be used to designate the time when a particular test was conducted
(e.g., The Session Request Delay at "TB" and <some specific UA interface>
was measured to be X ms.)

OR

we could add a Date-Time stamp format, and let TB and TS exist
in a locally-defined format, as long as their difference produces
sufficient resolution. There are plenty of date-time formats
that we could specify:
RFC 2330 specifies UTC format for time in section 6.1, when discussing
measurement units.
The IETF reference for time formats appears to be
http://tools.ietf.org/html/rfc1123#section-5.2.14,
but there are others such as ISO 8601.
NTP format has more resolution, but it is expressed
relative to 0h Jan 1 1900.

Session Setup: and Session Establishment:
The presence of these two definitions begs for a general definition of
"Session Failure", especially when so many metrics have "failure" versions.
The definition should probably include a reference to the IANA list
of Response codes (and thereby, the Reference for each code)
http://www.iana.org/assignments/sip-parameters

Singletons, Samples, Statistics (as defined in RFC 2330):
http://tools.ietf.org/html/rfc2330#section-11
Some one-line definitions for these terms are:
Singleton:  a single instance of a metric, a single result.
Sample: a number of single instances, or singletons.
Statistic: a metric derived from a sample by computing some statistic.

I think it will help to introduce these (or similar) definitions
into the SIP metrics draft (and the FRMWK). Each of the
message flows illustrate a Singleton metric, while equations like
the Average Registration Request Delay in section 3.1 are examples
of Statistical metrics.

The Data Collection Section 5.4 touches on the topic of Samples
in the last paragraph. Would it be useful to include discussion of how
the Sample should be collected for summarization with a Statistic?
Perhaps this could be addressed in a short section on Methodology?
(This is where it helps to associate a date-time with each
request-response singleton, so that the time coverage of the sample
and resulting statistic are known.)

Al

_______________________________________________
PMOL mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/pmol

Date: Tue, 29 Jan 2008 19:54:43 -0500
To: "Daryl Malas" <[email-masked]>, [email-masked]
From: Al Morton <[email-masked]>
Cc: 
Subject: [PMOL] SIP performance metrics draft

--===============0348309450==
Content-Type: text/html; charset="us-ascii"

<html>
<body>
Hi Daryl,<br><br>
Here are a few comments on the SIP draft.<br>
I'm raising some metric development related points,<br>
and trying to harmonize the draft with the pmol framework.<br><br>
Some of the points imply additional development of&nbsp;&nbsp; <br>
the pmol framework, and are comments for *both* drafts.<br>
(I've marked these with &quot;FRMWK&quot;)<br><br>
I will have a few more comments soon, but this is <br>
a good start.<br><br>
regards,<br>
Al<br><br>
===========================================================<br>
Intro<br><br>
It's useful to have a Scope, and the current statements of scope<br>
can be found in the Introduction section.&nbsp; I suggest re-titling
<br>
the section &quot;1.0&nbsp; Introduction and Scope&quot;, or splitting
into 2 sections.<br><br>
<br>
Active and/or Passive in scope of draft:<br>
I think this question came up along the way: <br>
Is the draft applicable to Active/Passive or both?<br>
I think the answer is both, and suggest to add this:
<dl>
<dd>The metrics defined in this document are applicable in scenarios
where the SIP messages launched (into a network under test) are dedicated
messages for testing purposes, or where the messages are user-initiated
and a portion of the live traffic present.&nbsp; These two scenarios are
sometimes referred to as active and passive measurement, respectively.
</dl>FRMWK: I'm not sure where to mention this consideration in the
text.<br>
&nbsp;<br>
Section 3.1 of the framework draft highlights the need to <br>
identify the audience, or users of the metrics <br>
(FRMWK: need to flesh this section out). <br>
Here's a suggestion, it could be combined with the paragraph<br>
that begins, &quot;the scope&quot;:
<dl>
<dd>The intended audience for this document can be found among network
operators, who often collect information on the responsiveness of the
network to customer requests for services.<br><br>
</dl>Section 3 candidate material to add<br><br>
SIP Configuration:&nbsp; Various timers influence performance, so
the<br>
draft must mention them (at least).&nbsp; Ideally, the supporting
info<br>
for any measurement report would include a list of relevant <br>
configuration variables and their settings (in the originating UA).<br>
T1 looks like a good candidate:<br>
(from
<a href="http://tools.ietf.org/html/rfc3261#section-17.1.1.1" eudora="autourl">
http://tools.ietf.org/html/rfc3261#section-17.1.1.1</a> )<br>
17.1.1.1 Overview of INVITE Transaction<br>
&nbsp;&nbsp; The INVITE transaction consists of a three-way
handshake.&nbsp; The client<br>
&nbsp;&nbsp; transaction sends an INVITE, the server transaction sends
responses,<br>
&nbsp;&nbsp; and the client transaction sends an ACK.&nbsp; For
unreliable transports<br>
&nbsp;&nbsp; (such as UDP), the client transaction retransmits requests
at an<br>
&nbsp;&nbsp; interval that starts at T1 seconds and doubles after
every<br>
&nbsp;&nbsp; retransmission.&nbsp; T1 is an estimate of the round-trip
time (RTT), and<br>
&nbsp;&nbsp; it defaults to 500 ms.&nbsp; Nearly all of the transaction
timers<br>
&nbsp;&nbsp; described here scale with T1, and changing T1 adjusts their
values.<br><br>
Elsewhere, the default value of T2 is 4 seconds. We usually list all
<br>
the configuration parameters in the BMWG drafts, so that everyone<br>
reports the same ones.<br><br>
<br>
Clock Accuracy and Error:&nbsp; Since most of the SIP metrics are
time<br>
intervals measured at a single UA, time sync between two remote<br>
points is not an issue. Time interval accuracy for the single <br>
clock should be identified as a source of error, however.<br>
Section 10 of RFC 2330
<a href="http://tools.ietf.org/html/rfc2330#section-10" eudora="autourl">
http://tools.ietf.org/html/rfc2330#section-10</a><br>
discusses clock issues in detail (reference in FRMWK).<br><br>
&quot;Wire Time&quot;: This is another issue described well in <br>
section 10.2 of RFC 2330
<a href="http://tools.ietf.org/html/rfc2330#section-10.2" eudora="autourl">
http://tools.ietf.org/html/rfc2330#section-10.2</a><br>
The basic problem is that there is a difference between the time<br>
when a host sees a packet for time-stamping, and the time that the<br>
packet actually appears on the &quot;wire&quot;.</body>
</html>

--===============0348309450==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

_______________________________________________
PMOL mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/pmol

--===============0348309450==--

Content-class: urn:content-classes:message
Subject: RE: [Sipping] [PMOL] SIP End-to-End Performance Metrics
Date: Thu, 24 Jan 2008 11:15:11 -0700
From: "Daryl Malas" <[email-masked]>
To: "Marian Delkinov" <[email-masked]>, "Hadriel Kaplan" <[email-masked]>
Cc: [email-masked], [email-masked]

Mario,

At this point, I agree with your assertion to keep SEER as is, unless
there is a strong argument to change it.

Thanks...

Daryl=20

-----Original Message-----
From: Marian Delkinov [mailto:[email-masked]]=20
Sent: Thursday, January 24, 2008 4:01 AM
To: Daryl Malas; Hadriel Kaplan
Cc: [email-masked]; [email-masked]
Subject: RE: [Sipping] [PMOL] SIP End-to-End Performance Metrics

Daryl,

Regarding adding 401/407 to the numerator of SEER, my opinion is that it
depends on the idea behind SEER definition.
1) If SEER should complement SER, but exclude the called party
behaviour, then I support keeping SEER as it is. This case matches the
NER definition in PSTN.=20
2) If SEER should represent the ability of the infrastructure to
establish sessions, then I would support adding 401/407 codes to the
numerator. (If you remember during our discussion over the last year, I
proposed adding all 4xx codes except 408 in the numerator of SEER, but
we then agreed that it would cover much larger scope than just excluding
the customer behaviour. Perhaps we have to define another metric to
cover those cases, if the WG supports that.) =20

Best regards!
Mario.

-----Original Message-----
From: Daryl Malas [mailto:[email-masked]]
Sent: Monday, 21 January, 2008 16:19
To: Marian Delkinov; Hadriel Kaplan
Cc: [email-masked]; [email-masked]
Subject: RE: [Sipping] [PMOL] SIP End-to-End Performance Metrics

Mario,

I agree.  This is the traditional difference between SER and SEER, much
like the difference between ASR and NER (ITU-T e.411).  Now, one thing
that might make sense is to add 401/407 to the numerator of the SEER
metric algorithm.

--Daryl

-----Original Message-----
From: Marian Delkinov [mailto:[email-masked]]
Sent: Monday, January 21, 2008 3:45 AM
To: Daryl Malas; Hadriel Kaplan
Cc: [email-masked]; [email-masked]
Subject: RE: [Sipping] [PMOL] SIP End-to-End Performance Metrics

Daryl and Hadriel,

I have some comments regarding SER and SEER:

> SER - need to specify that a 401/407 would also be subtracted in that=20
> denominator, or better yet the Invite would not be double-counted=20
> period.  Also, need to point out that "# of INVITE" is for=20
> new-sessions only (ie, ones without to-tags) - not re-Invites. (I know

> it's obvious, but ya never know)

If we go back to the analogy with PSTN and the ASR defined by ITU, I
would suggest leaving SER as it is. Thus the metric would really be
useful for estimating the %revenue compared to the total volume of
invites, regardless of the reason for failure (as it is in PSTN). For
analysis of the different aspects of the infrastructure efficiency I
suggest using complementary to SER metrics, as SEER (already defined),
and why not defining some more.

SEER: Section 3.7 defines SEER, however the formula still uses SER=3D...
I suggest using "SEER" in the formula in order to avoid confusion with
SER.

Best regards!
Mario.
=20

-----Original Message-----
From: Daryl Malas [mailto:[email-masked]]
Sent: Friday, 18 January, 2008 17:17
To: Hadriel Kaplan
Cc: [email-masked]; [email-masked]
Subject: Re: [Sipping] [PMOL] SIP End-to-End Performance Metrics

Hadriel,

Comments in-line...

On 1/17/08, Hadriel Kaplan <[email-masked]> wrote:
>
> Hey Daryl,
> Comments:
>
> Failed RRD - I'm a little confused by the wording of how we decide
what constitutes a failed registration.  Clearly a 401/407 challenge
round does not, but you state a second round would.  Why?  It's totally
possible it's not a failure.
> I'm also confused what the value/purpose of measuring this delay is.
> The section says measuring failed RRD is useful for detecting problems
with reaching the intended Registrar - that would be true if you counted
transaction timeouts, not request/response delays for auth failures, no?

[DM]
I agree this could create some false indications of failures.  I can
update the metric to only determine a failed condition based on a
timeout scenario.

Also, I'm going to put together a suggested list of metrics from this
draft.  The topic has come up several times on how many metrics should
be included.  Since, no one has suggested a list, I will throw one out.
>
> Failed SRD - need to specify a 401/407 is not a failed SRD (you say
any 4xx).
>
[DM]
I agree and will update the draft.

> Successful SDD - you say at the end "In these two examples, TB and TS
are the same even if the UAC/UAS receives the indicated messages instead
of sending them."  I'm pretty sure the delay in receiving a Bye and
sending the 200 ok for that is not really useful. :)  Or at least a very
different issue than that measured from sending a Bye and receiving a
200.
>
[DM]
I agree.  At one point, someone requested this to be exhaustive in all
conditions.  I can remove this to indicate relevance only in the
situation of initiating the Bye from either the UAC or UAS.

> Failed SDD - what do we count the time of a Bye to a 4xx (but not
401/407) or 5xx as?  Since those are the failure conditions for the
other cases, this seems a bit odd to no longer count it as one here.
[DM]
I'm not sure I understand this question.
>
> Failed SDT - this seems to be at odds with the definition of
successful SDT.  Successful SDT is stopped on sending the Bye, but this
one is stopped on timing out that sent Bye. (in other words, every call
will thus count as a successful SDT, and some will also count as a
failed one)  Also, what good is this failed SDT info?  You already have
Failed SDD.  It seems to me an SDT can only be of type "successful", and
the timer should be stopped at send/receipt of Bye.
[DM]
I agree.  I will update this to only be between the 200 and Bye.  It
should be the Bye regardless of whether or not it times out based on the
intiator.
>
> AHR - how does the UAC or UAS know this info?  I.e., how does the UAS
know what max-forwards the UAC started with, and how does the UAC know
what max-forwards the UAS received?
[DM]
The thought is the clients (or some other in-line monitoring device)
would have to capture this information at both ends and hold it in order
to correlate and make the determination.  This metric is a tricky one
and has come up much scrutiny.  It will work and could be very useful,
but will require some capture outside of SIP in all likelihood.
>
> SER - need to specify that a 401/407 would also be subtracted in that=20
> denominator, or better yet the Invite would not be double-counted=20
> period.  Also, need to point out that "# of INVITE" is for=20
> new-sessions only (ie, ones without to-tags) - not re-Invites. (I know

> it's obvious, but ya never know)
[DM]
Are you saying 401/407 should be included with this sentence "...to the
total number of attempted INVITE requests less INVITE requests resulting
in a 3XX response..."?

>
> -hadriel
>
> > -----Original Message-----
> > From: Daryl Malas [mailto:[email-masked]]
> > Sent: Friday, December 14, 2007 5:42 PM
> > To: [email-masked]
> > Cc: [email-masked]
> > Subject: [Sipping] [PMOL] SIP End-to-End Performance Metrics
> >
> > At the conference, I introduced the SIP End-to-End Performance=20
> > Metrics draft (draft-malas-performance-metrics-08) to the PMOL
working group.
> > Although this draft is referenced in the PMOL WG charter, I wanted=20
> > to ask everyone to review the draft and provide feedback of whether=20
> > or not you feel the current version is ready for the WG to accept as

> > a working group item.  A couple of questions came up, which I think=20
> > should be answered regarding this consideration:
> >
> > Is the SIPPING WG content with the current set of metrics?
> >
> > Are there too many?
> >
> > Do these metrics capture the relevant concerns of performance=20
> > regarding the SIP protocol?
> >
> > Are these metrics depicted accurately from a SIP protocol
perspective?
> >
> > In addition to these questions, I have a couple of tasks from the=20
> > PMOL group to include in the next revision.
> >
> > Here is a link to the draft:
> > http://www.ietf.org/internet-drafts/draft-malas-performance-metrics-
> > 08.txt
> >
> > Thanks...
> >
> > Daryl
> >
> >
> >
> > _______________________________________________
> > Sipping mailing list  https://www1.ietf.org/mailman/listinfo/sipping
> > This list is for NEW development of the application of SIP Use=20
> > [email-masked] for questions on current sip Use=20
> > [email-masked] for new developments of core SIP
>
> ______________________________________________________________________
> This email has been scanned by the MessageLabs Email Security System.
> For more information please visit http://www.messagelabs.com/email=20
> ______________________________________________________________________
>

_______________________________________________
Sipping mailing list  https://www1.ietf.org/mailman/listinfo/sipping
This list is for NEW development of the application of SIP Use
[email-masked] for questions on current sip Use
[email-masked] for new developments of core SIP

_______________________________________________
Sipping mailing list  https://www1.ietf.org/mailman/listinfo/sipping
This list is for NEW development of the application of SIP Use
[email-masked] for questions on current sip Use
[email-masked] for new developments of core SIP

_______________________________________________
PMOL mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/pmol

Content-class: urn:content-classes:message
Subject: RE: [Sipping] [PMOL] SIP End-to-End Performance Metrics
Date: Thu, 24 Jan 2008 12:00:37 +0100
From: "Marian Delkinov" <[email-masked]>
To: "Daryl Malas" <[email-masked]>, "Hadriel Kaplan" <[email-masked]>
Cc: [email-masked], [email-masked]

Daryl,

Regarding adding 401/407 to the numerator of SEER, my opinion is that it
depends on the idea behind SEER definition.
1) If SEER should complement SER, but exclude the called party
behaviour, then I support keeping SEER as it is. This case matches the
NER definition in PSTN.=20
2) If SEER should represent the ability of the infrastructure to
establish sessions, then I would support adding 401/407 codes to the
numerator. (If you remember during our discussion over the last year, I
proposed adding all 4xx codes except 408 in the numerator of SEER, but
we then agreed that it would cover much larger scope than just excluding
the customer behaviour. Perhaps we have to define another metric to
cover those cases, if the WG supports that.) =20

Best regards!
Mario.

-----Original Message-----
From: Daryl Malas [mailto:[email-masked]]=20
Sent: Monday, 21 January, 2008 16:19
To: Marian Delkinov; Hadriel Kaplan
Cc: [email-masked]; [email-masked]
Subject: RE: [Sipping] [PMOL] SIP End-to-End Performance Metrics

Mario,

I agree.  This is the traditional difference between SER and SEER, much
like the difference between ASR and NER (ITU-T e.411).  Now, one thing
that might make sense is to add 401/407 to the numerator of the SEER
metric algorithm.

--Daryl

-----Original Message-----
From: Marian Delkinov [mailto:[email-masked]]
Sent: Monday, January 21, 2008 3:45 AM
To: Daryl Malas; Hadriel Kaplan
Cc: [email-masked]; [email-masked]
Subject: RE: [Sipping] [PMOL] SIP End-to-End Performance Metrics

Daryl and Hadriel,

I have some comments regarding SER and SEER:

> SER - need to specify that a 401/407 would also be subtracted in that=20
> denominator, or better yet the Invite would not be double-counted=20
> period.  Also, need to point out that "# of INVITE" is for=20
> new-sessions only (ie, ones without to-tags) - not re-Invites. (I know

> it's obvious, but ya never know)

If we go back to the analogy with PSTN and the ASR defined by ITU, I
would suggest leaving SER as it is. Thus the metric would really be
useful for estimating the %revenue compared to the total volume of
invites, regardless of the reason for failure (as it is in PSTN). For
analysis of the different aspects of the infrastructure efficiency I
suggest using complementary to SER metrics, as SEER (already defined),
and why not defining some more.

SEER: Section 3.7 defines SEER, however the formula still uses SER=3D...
I suggest using "SEER" in the formula in order to avoid confusion with
SER.

Best regards!
Mario.
=20

-----Original Message-----
From: Daryl Malas [mailto:[email-masked]]
Sent: Friday, 18 January, 2008 17:17
To: Hadriel Kaplan
Cc: [email-masked]; [email-masked]
Subject: Re: [Sipping] [PMOL] SIP End-to-End Performance Metrics

Hadriel,

Comments in-line...

On 1/17/08, Hadriel Kaplan <[email-masked]> wrote:
>
> Hey Daryl,
> Comments:
>
> Failed RRD - I'm a little confused by the wording of how we decide
what constitutes a failed registration.  Clearly a 401/407 challenge
round does not, but you state a second round would.  Why?  It's totally
possible it's not a failure.
> I'm also confused what the value/purpose of measuring this delay is.
> The section says measuring failed RRD is useful for detecting problems
with reaching the intended Registrar - that would be true if you counted
transaction timeouts, not request/response delays for auth failures, no?

[DM]
I agree this could create some false indications of failures.  I can
update the metric to only determine a failed condition based on a
timeout scenario.

Also, I'm going to put together a suggested list of metrics from this
draft.  The topic has come up several times on how many metrics should
be included.  Since, no one has suggested a list, I will throw one out.
>
> Failed SRD - need to specify a 401/407 is not a failed SRD (you say
any 4xx).
>
[DM]
I agree and will update the draft.

> Successful SDD - you say at the end "In these two examples, TB and TS
are the same even if the UAC/UAS receives the indicated messages instead
of sending them."  I'm pretty sure the delay in receiving a Bye and
sending the 200 ok for that is not really useful. :)  Or at least a very
different issue than that measured from sending a Bye and receiving a
200.
>
[DM]
I agree.  At one point, someone requested this to be exhaustive in all
conditions.  I can remove this to indicate relevance only in the
situation of initiating the Bye from either the UAC or UAS.

> Failed SDD - what do we count the time of a Bye to a 4xx (but not
401/407) or 5xx as?  Since those are the failure conditions for the
other cases, this seems a bit odd to no longer count it as one here.
[DM]
I'm not sure I understand this question.
>
> Failed SDT - this seems to be at odds with the definition of
successful SDT.  Successful SDT is stopped on sending the Bye, but this
one is stopped on timing out that sent Bye. (in other words, every call
will thus count as a successful SDT, and some will also count as a
failed one)  Also, what good is this failed SDT info?  You already have
Failed SDD.  It seems to me an SDT can only be of type "successful", and
the timer should be stopped at send/receipt of Bye.
[DM]
I agree.  I will update this to only be between the 200 and Bye.  It
should be the Bye regardless of whether or not it times out based on the
intiator.
>
> AHR - how does the UAC or UAS know this info?  I.e., how does the UAS
know what max-forwards the UAC started with, and how does the UAC know
what max-forwards the UAS received?
[DM]
The thought is the clients (or some other in-line monitoring device)
would have to capture this information at both ends and hold it in order
to correlate and make the determination.  This metric is a tricky one
and has come up much scrutiny.  It will work and could be very useful,
but will require some capture outside of SIP in all likelihood.
>
> SER - need to specify that a 401/407 would also be subtracted in that=20
> denominator, or better yet the Invite would not be double-counted=20
> period.  Also, need to point out that "# of INVITE" is for=20
> new-sessions only (ie, ones without to-tags) - not re-Invites. (I know

> it's obvious, but ya never know)
[DM]
Are you saying 401/407 should be included with this sentence "...to the
total number of attempted INVITE requests less INVITE requests resulting
in a 3XX response..."?

>
> -hadriel
>
> > -----Original Message-----
> > From: Daryl Malas [mailto:[email-masked]]
> > Sent: Friday, December 14, 2007 5:42 PM
> > To: [email-masked]
> > Cc: [email-masked]
> > Subject: [Sipping] [PMOL] SIP End-to-End Performance Metrics
> >
> > At the conference, I introduced the SIP End-to-End Performance=20
> > Metrics draft (draft-malas-performance-metrics-08) to the PMOL
working group.
> > Although this draft is referenced in the PMOL WG charter, I wanted=20
> > to ask everyone to review the draft and provide feedback of whether=20
> > or not you feel the current version is ready for the WG to accept as

> > a working group item.  A couple of questions came up, which I think=20
> > should be answered regarding this consideration:
> >
> > Is the SIPPING WG content with the current set of metrics?
> >
> > Are there too many?
> >
> > Do these metrics capture the relevant concerns of performance=20
> > regarding the SIP protocol?
> >
> > Are these metrics depicted accurately from a SIP protocol
perspective?
> >
> > In addition to these questions, I have a couple of tasks from the=20
> > PMOL group to include in the next revision.
> >
> > Here is a link to the draft:
> > http://www.ietf.org/internet-drafts/draft-malas-performance-metrics-
> > 08.txt
> >
> > Thanks...
> >
> > Daryl
> >
> >
> >
> > _______________________________________________
> > Sipping mailing list  https://www1.ietf.org/mailman/listinfo/sipping
> > This list is for NEW development of the application of SIP Use=20
> > [email-masked] for questions on current sip Use=20
> > [email-masked] for new developments of core SIP
>
> ______________________________________________________________________
> This email has been scanned by the MessageLabs Email Security System.
> For more information please visit http://www.messagelabs.com/email=20
> ______________________________________________________________________
>

_______________________________________________
Sipping mailing list  https://www1.ietf.org/mailman/listinfo/sipping
This list is for NEW development of the application of SIP Use
[email-masked] for questions on current sip Use
[email-masked] for new developments of core SIP

_______________________________________________
Sipping mailing list  https://www1.ietf.org/mailman/listinfo/sipping
This list is for NEW development of the application of SIP Use
[email-masked] for questions on current sip Use
[email-masked] for new developments of core SIP

_______________________________________________
PMOL mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/pmol

Date: Wed, 23 Jan 2008 10:28:48 -0500
To: [email-masked]
From: Al Morton <[email-masked]>
Subject: [PMOL] Proposed Adoption of draft-malas-performance-metrics-08

PMOL Working Group:

A comment period for the Internet-Draft on

"SIP End-to-End Performance Metrics"

<draft-malas-performance-metrics-08.txt>

will be open from 23 January 2008 through 6 February 2003.

A URL for this Internet-Draft is:
http://tools.ietf.org/html/draft-malas-performance-metrics-08

The purpose of this comment period is to examine whether there is PMOL WG
consensus to adopt this draft as the basis for the SIP Performance Metrics
"Proof of Concept" work item described in the PMOL WG Charter,
http://www.ietf.org/html.charters/pmol-charter.html
and if so to progress this as a Working Group Draft.

STATUS

When this topic was raised at the IETF-70 PMOL session, there
was substantial readership and some support expressed to take-up
this version as the WG draft. There were also some
reservations expressed at the meeting about how well
the current draft matches the (evolving and not yet complete)
framework draft. Thus, completing this milestone will require
further development of the framework (and process) draft, as well.

Also, this draft has enjoyed interest and review of the SIPPING WG,
and members of the SIP development community continue to apply
their expertise to fine-tune the performance metric definitions.

Please weigh-in with any comments to this list or to the co-chairs:
<[email-masked]> and <[email-masked]>.

Al Morton
Alan Clark
pmol co-chairs

Note: The current version of the framework and process draft is here:
http://tools.ietf.org/html/draft-morton-perf-metrics-framework-01

_______________________________________________
PMOL mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/pmol

Content-class: urn:content-classes:message
Subject: RE: [Sipping] [PMOL] SIP End-to-End Performance Metrics
Date: Mon, 21 Jan 2008 08:19:25 -0700
From: "Daryl Malas" <[email-masked]>
To: "Marian Delkinov" <[email-masked]>, "Hadriel Kaplan" <[email-masked]>
Cc: [email-masked], [email-masked]

Mario,

I agree.  This is the traditional difference between SER and SEER, much
like the difference between ASR and NER (ITU-T e.411).  Now, one thing
that might make sense is to add 401/407 to the numerator of the SEER
metric algorithm.

--Daryl

-----Original Message-----
From: Marian Delkinov [mailto:[email-masked]]=20
Sent: Monday, January 21, 2008 3:45 AM
To: Daryl Malas; Hadriel Kaplan
Cc: [email-masked]; [email-masked]
Subject: RE: [Sipping] [PMOL] SIP End-to-End Performance Metrics

Daryl and Hadriel,

I have some comments regarding SER and SEER:

> SER - need to specify that a 401/407 would also be subtracted in that=20
> denominator, or better yet the Invite would not be double-counted=20
> period.  Also, need to point out that "# of INVITE" is for=20
> new-sessions only (ie, ones without to-tags) - not re-Invites. (I know

> it's obvious, but ya never know)

If we go back to the analogy with PSTN and the ASR defined by ITU, I
would suggest leaving SER as it is. Thus the metric would really be
useful for estimating the %revenue compared to the total volume of
invites, regardless of the reason for failure (as it is in PSTN). For
analysis of the different aspects of the infrastructure efficiency I
suggest using complementary to SER metrics, as SEER (already defined),
and why not defining some more.

SEER: Section 3.7 defines SEER, however the formula still uses SER=3D...
I suggest using "SEER" in the formula in order to avoid confusion with
SER.

Best regards!
Mario.
=20

-----Original Message-----
From: Daryl Malas [mailto:[email-masked]]
Sent: Friday, 18 January, 2008 17:17
To: Hadriel Kaplan
Cc: [email-masked]; [email-masked]
Subject: Re: [Sipping] [PMOL] SIP End-to-End Performance Metrics

Hadriel,

Comments in-line...

On 1/17/08, Hadriel Kaplan <[email-masked]> wrote:
>
> Hey Daryl,
> Comments:
>
> Failed RRD - I'm a little confused by the wording of how we decide
what constitutes a failed registration.  Clearly a 401/407 challenge
round does not, but you state a second round would.  Why?  It's totally
possible it's not a failure.
> I'm also confused what the value/purpose of measuring this delay is.
> The section says measuring failed RRD is useful for detecting problems
with reaching the intended Registrar - that would be true if you counted
transaction timeouts, not request/response delays for auth failures, no?

[DM]
I agree this could create some false indications of failures.  I can
update the metric to only determine a failed condition based on a
timeout scenario.

Also, I'm going to put together a suggested list of metrics from this
draft.  The topic has come up several times on how many metrics should
be included.  Since, no one has suggested a list, I will throw one out.
>
> Failed SRD - need to specify a 401/407 is not a failed SRD (you say
any 4xx).
>
[DM]
I agree and will update the draft.

> Successful SDD - you say at the end "In these two examples, TB and TS
are the same even if the UAC/UAS receives the indicated messages instead
of sending them."  I'm pretty sure the delay in receiving a Bye and
sending the 200 ok for that is not really useful. :)  Or at least a very
different issue than that measured from sending a Bye and receiving a
200.
>
[DM]
I agree.  At one point, someone requested this to be exhaustive in all
conditions.  I can remove this to indicate relevance only in the
situation of initiating the Bye from either the UAC or UAS.

> Failed SDD - what do we count the time of a Bye to a 4xx (but not
401/407) or 5xx as?  Since those are the failure conditions for the
other cases, this seems a bit odd to no longer count it as one here.
[DM]
I'm not sure I understand this question.
>
> Failed SDT - this seems to be at odds with the definition of
successful SDT.  Successful SDT is stopped on sending the Bye, but this
one is stopped on timing out that sent Bye. (in other words, every call
will thus count as a successful SDT, and some will also count as a
failed one)  Also, what good is this failed SDT info?  You already have
Failed SDD.  It seems to me an SDT can only be of type "successful", and
the timer should be stopped at send/receipt of Bye.
[DM]
I agree.  I will update this to only be between the 200 and Bye.  It
should be the Bye regardless of whether or not it times out based on the
intiator.
>
> AHR - how does the UAC or UAS know this info?  I.e., how does the UAS
know what max-forwards the UAC started with, and how does the UAC know
what max-forwards the UAS received?
[DM]
The thought is the clients (or some other in-line monitoring device)
would have to capture this information at both ends and hold it in order
to correlate and make the determination.  This metric is a tricky one
and has come up much scrutiny.  It will work and could be very useful,
but will require some capture outside of SIP in all likelihood.
>
> SER - need to specify that a 401/407 would also be subtracted in that=20
> denominator, or better yet the Invite would not be double-counted=20
> period.  Also, need to point out that "# of INVITE" is for=20
> new-sessions only (ie, ones without to-tags) - not re-Invites. (I know

> it's obvious, but ya never know)
[DM]
Are you saying 401/407 should be included with this sentence "...to the
total number of attempted INVITE requests less INVITE requests resulting
in a 3XX response..."?

>
> -hadriel
>
> > -----Original Message-----
> > From: Daryl Malas [mailto:[email-masked]]
> > Sent: Friday, December 14, 2007 5:42 PM
> > To: [email-masked]
> > Cc: [email-masked]
> > Subject: [Sipping] [PMOL] SIP End-to-End Performance Metrics
> >
> > At the conference, I introduced the SIP End-to-End Performance=20
> > Metrics draft (draft-malas-performance-metrics-08) to the PMOL
working group.
> > Although this draft is referenced in the PMOL WG charter, I wanted=20
> > to ask everyone to review the draft and provide feedback of whether=20
> > or not you feel the current version is ready for the WG to accept as

> > a working group item.  A couple of questions came up, which I think=20
> > should be answered regarding this consideration:
> >
> > Is the SIPPING WG content with the current set of metrics?
> >
> > Are there too many?
> >
> > Do these metrics capture the relevant concerns of performance=20
> > regarding the SIP protocol?
> >
> > Are these metrics depicted accurately from a SIP protocol
perspective?
> >
> > In addition to these questions, I have a couple of tasks from the=20
> > PMOL group to include in the next revision.
> >
> > Here is a link to the draft:
> > http://www.ietf.org/internet-drafts/draft-malas-performance-metrics-
> > 08.txt
> >
> > Thanks...
> >
> > Daryl
> >
> >
> >
> > _______________________________________________
> > Sipping mailing list  https://www1.ietf.org/mailman/listinfo/sipping
> > This list is for NEW development of the application of SIP Use=20
> > [email-masked] for questions on current sip Use=20
> > [email-masked] for new developments of core SIP
>
> ______________________________________________________________________
> This email has been scanned by the MessageLabs Email Security System.
> For more information please visit http://www.messagelabs.com/email=20
> ______________________________________________________________________
>

_______________________________________________
Sipping mailing list  https://www1.ietf.org/mailman/listinfo/sipping
This list is for NEW development of the application of SIP Use
[email-masked] for questions on current sip Use
[email-masked] for new developments of core SIP

_______________________________________________
Sipping mailing list  https://www1.ietf.org/mailman/listinfo/sipping
This list is for NEW development of the application of SIP Use
[email-masked] for questions on current sip Use
[email-masked] for new developments of core SIP

_______________________________________________
PMOL mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/pmol

Content-class: urn:content-classes:message
Subject: RE: [Sipping] [PMOL] SIP End-to-End Performance Metrics
Date: Mon, 21 Jan 2008 11:45:28 +0100
From: "Marian Delkinov" <[email-masked]>
To: "Daryl Malas" <[email-masked]>, "Hadriel Kaplan" <[email-masked]>
Cc: [email-masked], [email-masked]

Daryl and Hadriel,

I have some comments regarding SER and SEER:

> SER - need to specify that a 401/407 would also be subtracted in that=20
> denominator, or better yet the Invite would not be double-counted=20
> period.  Also, need to point out that "# of INVITE" is for=20
> new-sessions only (ie, ones without to-tags) - not re-Invites. (I know

> it's obvious, but ya never know)

If we go back to the analogy with PSTN and the ASR defined by ITU, I
would suggest leaving SER as it is. Thus the metric would really be
useful for estimating the %revenue compared to the total volume of
invites, regardless of the reason for failure (as it is in PSTN). For
analysis of the different aspects of the infrastructure efficiency I
suggest using complementary to SER metrics, as SEER (already defined),
and why not defining some more.

SEER: Section 3.7 defines SEER, however the formula still uses SER=3D...
I suggest using "SEER" in the formula in order to avoid confusion with
SER.

Best regards!
Mario.
=20

-----Original Message-----
From: Daryl Malas [mailto:[email-masked]]=20
Sent: Friday, 18 January, 2008 17:17
To: Hadriel Kaplan
Cc: [email-masked]; [email-masked]
Subject: Re: [Sipping] [PMOL] SIP End-to-End Performance Metrics

Hadriel,

Comments in-line...

On 1/17/08, Hadriel Kaplan <[email-masked]> wrote:
>
> Hey Daryl,
> Comments:
>
> Failed RRD - I'm a little confused by the wording of how we decide
what constitutes a failed registration.  Clearly a 401/407 challenge
round does not, but you state a second round would.  Why?  It's totally
possible it's not a failure.
> I'm also confused what the value/purpose of measuring this delay is.
> The section says measuring failed RRD is useful for detecting problems
with reaching the intended Registrar - that would be true if you counted
transaction timeouts, not request/response delays for auth failures, no?

[DM]
I agree this could create some false indications of failures.  I can
update the metric to only determine a failed condition based on a
timeout scenario.

Also, I'm going to put together a suggested list of metrics from this
draft.  The topic has come up several times on how many metrics should
be included.  Since, no one has suggested a list, I will throw one out.
>
> Failed SRD - need to specify a 401/407 is not a failed SRD (you say
any 4xx).
>
[DM]
I agree and will update the draft.

> Successful SDD - you say at the end "In these two examples, TB and TS
are the same even if the UAC/UAS receives the indicated messages instead
of sending them."  I'm pretty sure the delay in receiving a Bye and
sending the 200 ok for that is not really useful. :)  Or at least a very
different issue than that measured from sending a Bye and receiving a
200.
>
[DM]
I agree.  At one point, someone requested this to be exhaustive in all
conditions.  I can remove this to indicate relevance only in the
situation of initiating the Bye from either the UAC or UAS.

> Failed SDD - what do we count the time of a Bye to a 4xx (but not
401/407) or 5xx as?  Since those are the failure conditions for the
other cases, this seems a bit odd to no longer count it as one here.
[DM]
I'm not sure I understand this question.
>
> Failed SDT - this seems to be at odds with the definition of
successful SDT.  Successful SDT is stopped on sending the Bye, but this
one is stopped on timing out that sent Bye. (in other words, every call
will thus count as a successful SDT, and some will also count as a
failed one)  Also, what good is this failed SDT info?  You already have
Failed SDD.  It seems to me an SDT can only be of type "successful", and
the timer should be stopped at send/receipt of Bye.
[DM]
I agree.  I will update this to only be between the 200 and Bye.  It
should be the Bye regardless of whether or not it times out based on the
intiator.
>
> AHR - how does the UAC or UAS know this info?  I.e., how does the UAS
know what max-forwards the UAC started with, and how does the UAC know
what max-forwards the UAS received?
[DM]
The thought is the clients (or some other in-line monitoring device)
would have to capture this information at both ends and hold it in order
to correlate and make the determination.  This metric is a tricky one
and has come up much scrutiny.  It will work and could be very useful,
but will require some capture outside of SIP in all likelihood.
>
> SER - need to specify that a 401/407 would also be subtracted in that=20
> denominator, or better yet the Invite would not be double-counted=20
> period.  Also, need to point out that "# of INVITE" is for=20
> new-sessions only (ie, ones without to-tags) - not re-Invites. (I know

> it's obvious, but ya never know)
[DM]
Are you saying 401/407 should be included with this sentence "...to the
total number of attempted INVITE requests less INVITE requests resulting
in a 3XX response..."?

>
> -hadriel
>
> > -----Original Message-----
> > From: Daryl Malas [mailto:[email-masked]]
> > Sent: Friday, December 14, 2007 5:42 PM
> > To: [email-masked]
> > Cc: [email-masked]
> > Subject: [Sipping] [PMOL] SIP End-to-End Performance Metrics
> >
> > At the conference, I introduced the SIP End-to-End Performance=20
> > Metrics draft (draft-malas-performance-metrics-08) to the PMOL
working group.
> > Although this draft is referenced in the PMOL WG charter, I wanted=20
> > to ask everyone to review the draft and provide feedback of whether=20
> > or not you feel the current version is ready for the WG to accept as

> > a working group item.  A couple of questions came up, which I think=20
> > should be answered regarding this consideration:
> >
> > Is the SIPPING WG content with the current set of metrics?
> >
> > Are there too many?
> >
> > Do these metrics capture the relevant concerns of performance=20
> > regarding the SIP protocol?
> >
> > Are these metrics depicted accurately from a SIP protocol
perspective?
> >
> > In addition to these questions, I have a couple of tasks from the=20
> > PMOL group to include in the next revision.
> >
> > Here is a link to the draft:
> > http://www.ietf.org/internet-drafts/draft-malas-performance-metrics-
> > 08.txt
> >
> > Thanks...
> >
> > Daryl
> >
> >
> >
> > _______________________________________________
> > Sipping mailing list  https://www1.ietf.org/mailman/listinfo/sipping
> > This list is for NEW development of the application of SIP Use=20
> > [email-masked] for questions on current sip Use=20
> > [email-masked] for new developments of core SIP
>
> ______________________________________________________________________
> This email has been scanned by the MessageLabs Email Security System.
> For more information please visit http://www.messagelabs.com/email=20
> ______________________________________________________________________
>

_______________________________________________
Sipping mailing list  https://www1.ietf.org/mailman/listinfo/sipping
This list is for NEW development of the application of SIP Use
[email-masked] for questions on current sip Use
[email-masked] for new developments of core SIP

_______________________________________________
PMOL mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/pmol

Date: Fri, 18 Jan 2008 09:16:49 -0700
From: "Daryl Malas" <[email-masked]>
To: "Hadriel Kaplan" <[email-masked]>
Subject: Re: [Sipping] [PMOL] SIP End-to-End Performance Metrics
Cc: [email-masked], [email-masked]

Hadriel,

Comments in-line...

On 1/17/08, Hadriel Kaplan <[email-masked]> wrote:
>
> Hey Daryl,
> Comments:
>
> Failed RRD - I'm a little confused by the wording of how we decide what constitutes a failed registration.  Clearly a 401/407 challenge round does not, but you state a second round would.  Why?  It's totally possible it's not a failure.
> I'm also confused what the value/purpose of measuring this delay is.
> The section says measuring failed RRD is useful for detecting problems with reaching the intended Registrar - that would be true if you counted transaction timeouts, not request/response delays for auth failures, no?

[DM]
I agree this could create some false indications of failures.  I can
update the metric to only determine a failed condition based on a
timeout scenario.

Also, I'm going to put together a suggested list of metrics from this
draft.  The topic has come up several times on how many metrics should
be included.  Since, no one has suggested a list, I will throw one
out.
>
> Failed SRD - need to specify a 401/407 is not a failed SRD (you say any 4xx).
>
[DM]
I agree and will update the draft.

> Successful SDD - you say at the end "In these two examples, TB and TS are the same even if the UAC/UAS receives the indicated messages instead of sending them."  I'm pretty sure the delay in receiving a Bye and sending the 200 ok for that is not really useful. :)  Or at least a very different issue than that measured from sending a Bye and receiving a 200.
>
[DM]
I agree.  At one point, someone requested this to be exhaustive in all
conditions.  I can remove this to indicate relevance only in the
situation of initiating the Bye from either the UAC or UAS.

> Failed SDD - what do we count the time of a Bye to a 4xx (but not 401/407) or 5xx as?  Since those are the failure conditions for the other cases, this seems a bit odd to no longer count it as one here.
[DM]
I'm not sure I understand this question.
>
> Failed SDT - this seems to be at odds with the definition of successful SDT.  Successful SDT is stopped on sending the Bye, but this one is stopped on timing out that sent Bye. (in other words, every call will thus count as a successful SDT, and some will also count as a failed one)  Also, what good is this failed SDT info?  You already have Failed SDD.  It seems to me an SDT can only be of type "successful", and the timer should be stopped at send/receipt of Bye.
[DM]
I agree.  I will update this to only be between the 200 and Bye.  It
should be the Bye regardless of whether or not it times out based on
the intiator.
>
> AHR - how does the UAC or UAS know this info?  I.e., how does the UAS know what max-forwards the UAC started with, and how does the UAC know what max-forwards the UAS received?
[DM]
The thought is the clients (or some other in-line monitoring device)
would have to capture this information at both ends and hold it in
order to correlate and make the determination.  This metric is a
tricky one and has come up much scrutiny.  It will work and could be
very useful, but will require some capture outside of SIP in all
likelihood.
>
> SER - need to specify that a 401/407 would also be subtracted in that denominator, or better yet the Invite would not be double-counted period.  Also, need to point out that "# of INVITE" is for new-sessions only (ie, ones without to-tags) - not re-Invites. (I know it's obvious, but ya never know)
[DM]
Are you saying 401/407 should be included with this sentence "...to
the total number of attempted INVITE requests less INVITE requests
resulting in a 3XX response..."?

>
> -hadriel
>
> > -----Original Message-----
> > From: Daryl Malas [mailto:[email-masked]]
> > Sent: Friday, December 14, 2007 5:42 PM
> > To: [email-masked]
> > Cc: [email-masked]
> > Subject: [Sipping] [PMOL] SIP End-to-End Performance Metrics
> >
> > At the conference, I introduced the SIP End-to-End Performance Metrics
> > draft (draft-malas-performance-metrics-08) to the PMOL working group.
> > Although this draft is referenced in the PMOL WG charter, I wanted to
> > ask everyone to review the draft and provide feedback of whether or not
> > you feel the current version is ready for the WG to accept as a working
> > group item.  A couple of questions came up, which I think should be
> > answered regarding this consideration:
> >
> > Is the SIPPING WG content with the current set of metrics?
> >
> > Are there too many?
> >
> > Do these metrics capture the relevant concerns of performance regarding
> > the SIP protocol?
> >
> > Are these metrics depicted accurately from a SIP protocol perspective?
> >
> > In addition to these questions, I have a couple of tasks from the PMOL
> > group to include in the next revision.
> >
> > Here is a link to the draft:
> > http://www.ietf.org/internet-drafts/draft-malas-performance-metrics-08.txt
> >
> > Thanks...
> >
> > Daryl
> >
> >
> >
> > _______________________________________________
> > Sipping mailing list  https://www1.ietf.org/mailman/listinfo/sipping
> > This list is for NEW development of the application of SIP
> > Use [email-masked] for questions on current sip
> > Use [email-masked] for new developments of core SIP
>
> ______________________________________________________________________
> This email has been scanned by the MessageLabs Email Security System.
> For more information please visit http://www.messagelabs.com/email
> ______________________________________________________________________
>

_______________________________________________
PMOL mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/pmol

Subject: [PMOL] [Sipping] Draft Change of Email Address
From: Daryl Malas <[email-masked]>
To: [email-masked]
Date: Thu, 03 Jan 2008 17:02:19 -0700

As of January 4, 2008, this email address ([email-masked]) and
[email-masked] will no longer be a valid address to reach me
regarding comments or questions for the following two drafts:

(SIP End-to-End Performance Metrics)
draft-malas-performance-metrics-08

(Session Initiation Protocol (SIP) Overload Control)
draft-hilt-sipping-overload-03

Please use the interim email address of [email-masked] until further
notice.

Thank you,

Daryl Malas

_______________________________________________
PMOL mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/pmol

From: "Alan Clark" <[email-masked]>
To: <[email-masked]>
Date: Thu, 3 Jan 2008 15:55:05 -0500
Subject: [PMOL] Draft minutes posted

This is a multi-part message in MIME format.

--===============1638233590==
Content-Type: multipart/alternative;
	boundary="----=_NextPart_000_00D8_01C84E21.02BFA320"

This is a multi-part message in MIME format.

------=_NextPart_000_00D8_01C84E21.02BFA320
Content-Type: text/plain;
	charset="us-ascii"
Content-Transfer-Encoding: 7bit

The minutes for the December PMOL meeting have been posted to: 

http://www3.ietf.org/proceedings/07dec/minutes/pmol.txt

Please let me know if you have any comments or corrections (prior to 7th
Jan)

Happy New Year!

Alan

------=_NextPart_000_00D8_01C84E21.02BFA320
Content-Type: text/html;
	charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

<html xmlns:o=3D"urn:schemas-microsoft-com:office:office" =
xmlns:w=3D"urn:schemas-microsoft-com:office:word" =
xmlns=3D"http://www.w3.org/TR/REC-html40">

<head>
<META HTTP-EQUIV=3D"Content-Type" CONTENT=3D"text/html; =
charset=3Dus-ascii">
<meta name=3DGenerator content=3D"Microsoft Word 11 (filtered medium)">
<style>
<!--
 /* Style Definitions */
 p.MsoNormal, li.MsoNormal, div.MsoNormal
	{margin:0in;
	margin-bottom:.0001pt;
	font-size:12.0pt;
	font-family:"Times New Roman";}
a:link, span.MsoHyperlink
	{color:blue;
	text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
	{color:purple;
	text-decoration:underline;}
span.EmailStyle17
	{mso-style-type:personal-compose;
	font-family:Arial;
	color:windowtext;}
@page Section1
	{size:8.5in 11.0in;
	margin:1.0in 1.25in 1.0in 1.25in;}
div.Section1
	{page:Section1;}
-->
</style>

</head>

<body lang=3DEN-US link=3Dblue vlink=3Dpurple>

<div class=3DSection1>

<p class=3DMsoNormal><font size=3D2 face=3DArial><span =
style=3D'font-size:10.0pt;
font-family:Arial'>The minutes for the December PMOL meeting have been =
posted
to: <o:p></o:p></span></font></p>

<p class=3DMsoNormal><font size=3D2 face=3DArial><span =
style=3D'font-size:10.0pt;
font-family:Arial'><o:p>&nbsp;</o:p></span></font></p>

<p class=3DMsoNormal><font size=3D2 face=3DArial><span =
style=3D'font-size:10.0pt;
font-family:Arial'>http://www3.ietf.org/proceedings/07dec/minutes/pmol.tx=
t<o:p></o:p></span></font></p>

<p class=3DMsoNormal><font size=3D2 face=3DArial><span =
style=3D'font-size:10.0pt;
font-family:Arial'><o:p>&nbsp;</o:p></span></font></p>

<p class=3DMsoNormal><font size=3D2 face=3DArial><span =
style=3D'font-size:10.0pt;
font-family:Arial'>Please let me know if you have any comments or =
corrections
(prior to 7<sup>th</sup> Jan)<o:p></o:p></span></font></p>

<p class=3DMsoNormal><font size=3D2 face=3DArial><span =
style=3D'font-size:10.0pt;
font-family:Arial'><o:p>&nbsp;</o:p></span></font></p>

<p class=3DMsoNormal><font size=3D2 face=3DArial><span =
style=3D'font-size:10.0pt;
font-family:Arial'>Happy New Year!<o:p></o:p></span></font></p>

<p class=3DMsoNormal><font size=3D2 face=3DArial><span =
style=3D'font-size:10.0pt;
font-family:Arial'><o:p>&nbsp;</o:p></span></font></p>

<p class=3DMsoNormal><font size=3D2 face=3DArial><span =
style=3D'font-size:10.0pt;
font-family:Arial'>Alan<o:p></o:p></span></font></p>

<p class=3DMsoNormal><font size=3D2 face=3DArial><span =
style=3D'font-size:10.0pt;
font-family:Arial'><o:p>&nbsp;</o:p></span></font></p>

</div>

</body>

</html>

------=_NextPart_000_00D8_01C84E21.02BFA320--

--===============1638233590==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

_______________________________________________
PMOL mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/pmol

--===============1638233590==--