Subject: [Pce] Stateful PCE: inconsistency in "resource limit" text

From: Jonathan Hardwick <[email-masked]>
Date: Mon, 8 May 2017 16:19:08 +0000

Hi PCE WG

I've been tidying up the stateful PCE draft to prepare it for publication and I have discovered an inconsistency in how the stateful PCE is supposed to handle an overflow of its per-PCC resource limit.  In section 5.6 it says:

   A PCE implementing a limit on the resources a single PCC can occupy,
   MUST send a PCNtf message with Notification Type to be allocated by
   IANA (Stateful PCE resource limit exceeded) and Notification Value to
   be allocated by IANA (Entering resource limit exceeded state) in
   response to the PCRpt message triggering this condition in the
   synchronization phase and MUST terminate the session.

Whereas in section 6.1 it says:

   A PCE may choose to implement a limit on the resources a single PCC

   can occupy.  If a PCRpt is received that causes the PCE to exceed

   this limit, the PCE MUST notify the PCC using a PCNtf message with

   Notification Type to be allocated by IANA (Stateful PCE resource

   limit exceeded) and Notification Value to be allocated by IANA

   (Entering resource limit exceeded state) and MAY terminate the

   session.

These sections are inconsistent because the first says the PCE MUST terminate the session whereas the second says the PCE MAY terminate the session.

Furthermore, in section 8.6, the following notification is defined for "exiting resource limit exceeded state", but this notification is not referenced anywhere in the text.

    Notification-Type  Meaning

       4        Stateful PCE resource limit exceeded

                 Notification-value=2:   Exiting resource limit exceeded

                                         state

Please could I ask all implementers:

-        MUST the PCE terminate the session if its state limit is exceeded, or MAY it leave it open?

-        Has anybody implemented the "exiting resource limit exceeded state" notification?  If so, how are you using it?

Your swiftest answers would be very much appreciated!

If I don't get any contradictory replies, my default action will be to say that the session MUST be terminated and to remove the unreferenced notification-value.

Many thanks
Jon

---

From: Ramon Casellas <[email-masked]>
Date: Mon, 8 May 2017 20:11:01 +0200

Hi all,

FWIW
> -MUST the PCE terminate the session if its state limit is exceeded, or 
> MAY it leave it open?
>
Honestly, I am not 100% sure, although I may be more in favor of : MUST 
terminate the session

Rationale:
I am assuming resources in the"pce" process (as in memory, CPU cycles ... )

1) If it is a soft-limit or a PCE transient state, it MAY leave it open. 
Leaving it open suggests that the PCC can retry / resend at a later 
stage that PCRpt in order to hopefully complete the sync. That would 
mean using PCNtf as NACK - maybe not adequate.

2) if it is a hard limit (as it seems to be the case -- note that it is 
not stated how a PCC can discover its limits --) leaving it open and the 
PCNtf it tells the PCC it hit a limit. The PCC could proceed assuming 
that it can report "up to N" and consider the sync done. Deducing this 
by repeatedly setting up sessions is IMHO more complex. However, not 
sure of the implications of having partial LSPDB sync

3) Re-reading the relevant sections,  error cases in the section seem to 
explicitly state "MUST close the session". If hitting a hard limit is an 
error, it MUST terminate the session (uniform). The question then is 
"why is the PCE sending a PCNtf and not a PCErr in that case"? AFAIK 
PCNtf seems to be more permissive.

> -Has anybody implemented the “exiting resource limit exceeded state” 
> notification?  If so, how are you using it?
>
No. Currently, we only track the "number of LSPs reported per PCC" but 
without actual per PCC limits

>
> If I don’t get any contradictory replies, my default action will be to 
> say that the session MUST be terminated and to remove the unreferenced 
> notification-value.
>
Would you consider changing the PCNtf to PCErr ?

Regards
Ramon

---

From: Cyril Margaria <[email-masked]>
Date: Mon, 8 May 2017 17:09:39 -0400

Hi,

>From what I recall, the limit exceeded can refer to the number of LSPs,
memory, ..etc and the notification was introduced to support the same logic
as rfc5440 "Overloaded PCE" notification.

To keep that and to support soft/administrative limits, I am in favor of
MAY terminate the session. If the Peer decides to terminate the session, a
specific code must be used, otherwise the other peer will reconnect and the
session will keep flapping.

BR,
Cyril

On 8 May 2017 at 12:19, Jonathan Hardwick <[email-masked]>
wrote:

> Hi PCE WG
>
>
>
> I’ve been tidying up the stateful PCE draft to prepare it for publication
> and I have discovered an inconsistency in how the stateful PCE is supposed
> to handle an overflow of its per-PCC resource limit.  In section 5.6 it
> says:
>
>
>
> These sections are inconsistent because the first says the PCE MUST
> terminate the session whereas the second says the PCE MAY terminate the
> session.
>
>
>
> Furthermore, in section 8.6, the following notification is defined for
> “exiting resource limit exceeded state”, but this notification is not
> referenced anywhere in the text.
>
>
>
> -        Has anybody implemented the “exiting resource limit exceeded
> state” notification?  If so, how are you using it?
>
>
>
> If I don’t get any contradictory replies, my default action will be to say
> that the session MUST be terminated and to remove the unreferenced
> notification-value.
>
>
>
> Many thanks
>
> Jon
>
>
>
> _______________________________________________
> Pce mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/pce
>
>

---

From: Julien Meuric <[email-masked]>
Date: Tue, 9 May 2017 10:35:25 +0200

Hi Ramon,

Indeed, the I-D used to consider an error for this case, but RFC 5440
gives the pace and 2 codepoints were allocated:
- PCErr are associated to PCEP issues ("when a protocol error condition
is met or when the request is not compliant with the PCEP specification");
- PCNtf are typically used for other cases, including PCE state (e.g.
overload) and policy (e.g. threshold).

Please note also that the current (orphan) text of the I-D allows to use
the PCNtf for "Exiting resource limit exceeded state". I am not saying
the latter should be kept, but considering this option sustains the idea
of having 2 possible messages/behaviors in PCEP.

Thanks for your feedback,

Julien

May. 08, 2017 - [email-masked]:
> Hi, 
>
> From what I recall, the limit exceeded can refer to the number of
> LSPs, memory, ..etc and the notification was introduced to support the
> same logic as rfc5440 "Overloaded PCE" notification.
>
> To keep that and to support soft/administrative limits, I am in favor
> of MAY terminate the session. If the Peer decides to terminate the
> session, a specific code must be used, otherwise the other peer will
> reconnect and the session will keep flapping.
>
> BR,
> Cyril
>
> On 8 May 2017 at 12:19, Jonathan Hardwick
> <[email-masked]
> <mailto:[email-masked]>> wrote:
>
>     Hi PCE WG
>
>      
>
>     I’ve been tidying up the stateful PCE draft to prepare it for
>     publication and I have discovered an inconsistency in how the
>     stateful PCE is supposed to handle an overflow of its per-PCC
>     resource limit.  In section 5.6 it says:
>
>      
>
>     These sections are inconsistent because the first says the PCE
>     MUST terminate the session whereas the second says the PCE MAY
>     terminate the session.
>
>      
>
>     Furthermore, in section 8.6, the following notification is defined
>     for “exiting resource limit exceeded state”, but this notification
>     is not referenced anywhere in the text.
>
>      
>
>     -        Has anybody implemented the “exiting resource limit
>     exceeded state” notification?  If so, how are you using it?
>
>      
>
>     If I don’t get any contradictory replies, my default action will
>     be to say that the session MUST be terminated and to remove the
>     unreferenced notification-value.
>
>      
>
>     Many thanks
>
>     Jon
>
>      
>
>     _______________________________________________ Pce mailing list
>     [email-masked] <mailto:[email-masked]>
>     <https://www.ietf.org/mailman/listinfo/pce> 
>

---

From: Ramon Casellas <[email-masked]>
Date: Tue, 9 May 2017 10:50:06 +0200

> of having 2 possible messages/behaviors in PCEP.

Hi Julien,

This is indeed making me raise more questions than expected.

- Reading the section I got the feeling that any event preventing to 
reach full sync state caused a PCErr (now PCNtf) and a MUST session 
close. was it the intent?

- As a minor comment, I see your point on the PCErr / PCNtf "macroscopic 
use", although I would humbly object, notably in view of the whole PCErr 
5: policy violation, that includes cases such as "global concurrent 
optimization not allowed" or "objective function not allowed" so, while 
I can agree with your view, IMHO it is not black/white.
In particular, I think you are not fully quoting RFC5440

    The PCEP Error message (also referred to as a PCErr message) is sent
    in several situations: when a protocol error condition is met or when
    the request is not compliant with the PCEP specification (e.g.,
    reception of a malformed message, reception of a message with a
    mandatory missing object, policy violation, unexpected message,
    unknown request reference)

It seems to me that policiy violation is a PCErr.  However, the main 
question is:

- Is not reaching full sync considered a protocol error? (for whatever 
reason, and beyond a single message exchange but more as a transactional 
thing/sequence)  If yes, I would vote for a PCErr / MUST close

- If it is not an error (I am not fully aware of the implications) then 
we should allow PCNtf + MAY leave it open. Otherwise, my view remains 
PCErr + MUST close.

Thanks!
Ramon

---

From: Dhruv Dhody <[email-masked]>
Date: Tue, 9 May 2017 09:29:36 +0000

Hi Jon, WG,

I feel this was done to differentiate two cases -

(1)   The resource limit happened during state synchronization (section 5.6) - with MUST close session

(2)   The resource limit happened during normal state report - with MAY close session

We could make this explicit and differentiate the two scenarios?

Regards,
Dhruv

From: Pce [mailto:[email-masked]] On Behalf Of Jonathan Hardwick
Sent: 08 May 2017 21:49
To: [email-masked]
Subject: [Pce] Stateful PCE: inconsistency in "resource limit" text

Hi PCE WG

   session.

                                         state

Many thanks
Jon

---

From: Julien Meuric <[email-masked]>
Date: Tue, 9 May 2017 12:18:26 +0200

Hi again Ramon,

We are tackling 2 topics here: the stateful I-D on the table and PCErr
vs. PCNtf at large (see [JM] below).

On the former, we must not forget that:
- the use of PCNtf is consistent with the overload case in RFC 5440,
- draft-ietf-pce-stateful-pce passed IESG review (as well as previous WG
and IETF LCs),
- draft-ietf-pce-stateful-pce has early allocated codepoints.
As a result, the PCNtf is not an open question in the current case.

Then in the text...

> close. was it the intent?

[JM] Allow me to rephrase a bit: "preventing to reach full sync state
caused an error advertised using a PCNtf". The impact on session closure
is the only discussed topic.

>
> - As a minor comment, I see your point on the PCErr / PCNtf
> "macroscopic use", although I would humbly object, notably in view of
> the whole PCErr 5: policy violation, that includes cases such as
> "global concurrent optimization not allowed" or "objective function
> not allowed" so, while I can agree with your view, IMHO it is not
> black/white.
> It seems to me that policiy violation is a PCErr.  

[JM] You are right, it is not black/white. E.g. there are various types
of policy violations:
- operations the PCEP peer is not allowed to do (it may even be aware
before trying);
- live situation changes, like overload or threshold crossing (here, a
PCC is not necessarily misbehaving but hitting a PCE-local value);
- etc.
Generally speaking, I do not see in RFC 5440 a very strict line
splitting between PCErr and PCNtf, the discussion on ties should happen
on a case by case basis. We should at least avoid to decide based on the
message name. The actual question is: does a given feature match the
5/12 message/object kind or the 6/13 one? An opportunity (even if not
specified) to exit a faced situation (like overload or threshold
crossing) may often suggest to consider PCNtf, but this is not a strict
definition.

> However, the main question is:
>
> reason, and beyond a single message exchange but more as a
> transactional thing/sequence)  If yes, I would vote for a PCErr / MUST
> close
>
> - If it is not an error (I am not fully aware of the implications)
> then we should allow PCNtf + MAY leave it open. Otherwise, my view
> remains PCErr + MUST close.

[JM] I do not think message type and session closure are so tied. In the
current document, PCNtf is no more questionable. Consequently, can we
consider your statement as a voice for "MAY" in the document?

Thanks,

Julien

---

From: Julien Meuric <[email-masked]>
Date: Tue, 9 May 2017 12:20:41 +0200

Hi Dhruv,

At this stage, it seems a bit late to introduce this change into the
document. However, keeping only "MAY" would allow implementations to
behave the way you suggest. Do we consider your feedback as supporting
"MAY"?

Thanks,

Julien

May. 09, 2017 - [email-masked]:
>
> Hi Jon, WG,
>
>  
>
> I feel this was done to differentiate two cases –
>
>  
>
> (1)   The resource limit happened during state synchronization
> (section 5.6) – with MUST close session
>
> (2)   The resource limit happened during normal state report – with
> MAY close session
>
>  
>
> *From:*Pce [mailto:[email-masked]] *On Behalf Of *Jonathan Hardwick
> *Sent:* 08 May 2017 21:49
>
>  
>
> Hi PCE WG
>
>  
>
> publication and I have discovered an inconsistency in how the stateful
> PCE is supposed to handle an overflow of its per-PCC resource limit. 
> In section 5.6 it says:
>
>  
>
> terminate the session whereas the second says the PCE MAY terminate
> the session.
>
>  
>

---

From: Ramon Casellas <[email-masked]>
Date: Tue, 9 May 2017 12:50:22 +0200

> As a result, the PCNtf is not an open question in the current case.
(snip)

Hi again Julien,

Thanks for reminding me of the specific question and sorry for diverging 
in multiple ways :) In view of your further comments and draft 
constraints, my preference remains as follows:

I still consider not being able to complete sync as a serious error and 
I would suggest a MUST close the session, regardless of the actual 
message to indicate such failure.

Thanks
R.

---

From: Dhruv Dhody <[email-masked]>
Date: Tue, 9 May 2017 11:19:47 +0000

Hi Julien, 

I agree. So "MAY" is the best way forward. 

Regards,
Dhruv

> -----Original Message-----
> From: Julien Meuric [mailto:[email-masked]]
> Sent: 09 May 2017 15:51
> To: Dhruv Dhody <[email-masked]>
> Cc: Jonathan Hardwick <[email-masked]>; [email-masked]
> Subject: Re: [Pce] Stateful PCE: inconsistency in "resource limit" text
> 
> Hi Dhruv,
> 
> > (section 5.6) - with MUST close session
> >
> > I've been tidying up the stateful PCE draft to prepare it for
> > "exiting resource limit exceeded state", but this notification is not
> > -          Has anybody implemented the "exiting resource limit
> > exceeded state" notification?  If so, how are you using it?
> >
> >
> >
> > If I don't get any contradictory replies, my default action will be to

---

From: Julien Meuric <[email-masked]>
Date: Tue, 9 May 2017 14:02:41 +0200

Ramon,

Thanks a lot for your feedback, this is very helpful.

Julien

---

From: <[email-masked]>
Date: Wed, 10 May 2017 06:56:59 +0000

Hi,

I think the main point is what does the session reset bring here ? IMO nothing in that case, your session will constantly bounce until the someone reduce the number of states sent by a particular PCC which will create even more load on the PCE. This is an event that cannot be corrected by the session reset.
Logging the Notification is a better approach that would allow the operator to be informed and trigger a fix.

So I do not think that "MUST terminate" is a good way to go.

Brgds,

Stephane

-----Original Message-----
From: Pce [mailto:[email-masked]] On Behalf Of Julien Meuric
Sent: Tuesday, May 09, 2017 12:21
To: Dhruv Dhody
Cc: [email-masked]
Subject: Re: [Pce] Stateful PCE: inconsistency in "resource limit" text

Hi Dhruv,

At this stage, it seems a bit late to introduce this change into the document. However, keeping only "MAY" would allow implementations to behave the way you suggest. Do we consider your feedback as supporting "MAY"?

Thanks,

Julien

_________________________________________________________________________________________________________________________

Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc
pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler
a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration,
Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci.

This message and its attachments may contain confidential or privileged information that may be protected by law;
they should not be distributed, used or copied without authorisation.
If you have received this email in error, please notify the sender and delete this message and its attachments.
As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified.
Thank you.

---

From: Ramon Casellas <[email-masked]>
Date: Wed, 10 May 2017 09:08:06 +0200

> I think the main point is what does the session reset bring here ? IMO nothing in that case, your session will constantly bounce until the someone reduce the number of states sent by a particular P…
I don't have a strong opinion after all, but before closing the session 
you get the proper PCNtf. There is no need to constantly bounce. To your 
question, IMHO closing the session accomplishes precisely 
(deterministically) that either you reach end of sync or not; not 
allowing to get a "half-sync" state.

> Logging the Notification is a better approach that would allow the operator to be informed and trigger a fix.
You log it anyway. The session is not "abruptly" closed.

At the end, I am fine with either choice. Even if I consider not being 
able to complete sync an issue, I also consider setting a policy that 
prevents a pcc to sync another (unrelated) issue.

Regards
Ramon

---

From: Robert Varga <[email-masked]>
Date: Wed, 17 May 2017 13:51:37 +0200

> close. was it the intent?

Hello Ramon,

with a co-author hat on, but without loading the draft completely into
brain again, yes, this was the intent. The reasoning behind is to
provide an initial baseline for the state present on the PCC, agreed by
both PCE and PCC.

This simplifies the protocol design a bit, as we do not have to deal
with state synchronization being half-done.

Furthermore it gives the PCE a chance to attempt to re-negotiate the
session parameters based on the problem it has seen with the PCRpt.

Regards,
Robert

---

From: Jonathan Hardwick <[email-masked]>
Date: Wed, 17 May 2017 13:37:08 +0000

Hi all

Thanks for your feedback on this issue.  I think we are probably in a position to close this issue down.  To summarize:

- The original intent was that the PCE MUST close the session.
- It seems that nobody has implemented the "exiting resource limit exceeded state" notification.

On the other hand, if we did weaken "MUST close" to "MAY close", then the draft provides no guidance about what the PCC and PCE are supposed to do next with this session in which only part of the state has been kept by the PCE.  I don't want to start drafting that guidance at this late stage.

My conclusion is that we should specify that the PCE MUST close the session, and we should release the code point currently allocated to the "exiting resource limit exceeded state" notification.

If anyone has strong objections to this, please shout ASAP.

Many thanks
Jon

-----Original Message-----
From: Pce [mailto:[email-masked]] On Behalf Of Robert Varga
Sent: 17 May 2017 12:52
To: Ramon Casellas <[email-masked]>; [email-masked]
Subject: Re: [Pce] Stateful PCE: inconsistency in "resource limit" text

Hello Ramon,

with a co-author hat on, but without loading the draft completely into brain again, yes, this was the intent. The reasoning behind is to provide an initial baseline for the state present on the PCC, agreed by both PCE and PCC.

This simplifies the protocol design a bit, as we do not have to deal with state synchronization being half-done.

Furthermore it gives the PCE a chance to attempt to re-negotiate the session parameters based on the problem it has seen with the PCRpt.

Regards,
Robert