Subject: Re: [OSPF] Solicit feedbacks on draft-dong-ospf-maxage-flush-problem-statement

From: "Les Ginsberg (ginsberg)" <[email-masked]>
Date: Mon, 1 Aug 2016 07:08:44 +0000

Jie -

Fully agree that IS-IS and OSPF differ in this regard.

https://www.ietf.org/id/draft-ietf-isis-remaining-lifetime-01.txt addresses problems where corruption of the remaining lifetime occurs either during transmission/reception or due to some DOS attack. This isn't a concern w OSPF (hope you agree).

What remains is the possibility that an implementation has some bug and unintentionally modifies the age to something other than what it should be due to the actual elapsed time since LSA generation. I suppose a mechanism equivalent to what the IS-IS draft defined i.e. setting the age to "new" (0 in OSPF case) when first receiving a non-self-generated LSA could be useful to prevent negative impacts of such an implementation bug. Is this what you intend?

As written, the draft makes claims that are at least misleading - and I believe actually incorrect. In Section 6 you say:

"The LS age field may be altered as a result of
   packet corruption, such modification cannot be detected by LSA
   checksum nor OSPF packet cryptographic authentication."

This isn't correct.

What would be helpful - at least to me - is to move from a generic problem statement to the specific problem you want to solve and the proposed solution. This also requires you to more clearly state the cases where there is an actual vulnerability. It would be a lot easier to support the draft if this were done.

   Les

From: Dongjie (Jimmy) [mailto:[email-masked]]
Sent: Sunday, July 31, 2016 11:48 PM
To: Les Ginsberg (ginsberg); [email-masked]
Cc: Zhangxudong (zhangxudong, VRP); [email-masked]
Subject: RE: [OSPF] Solicit feedbacks on draft-dong-ospf-maxage-flush-problem-statement

Hi Les,

Thanks for your comments.

OSPF packet level checksum and authentication can only protect the assembled LSU packet one hop on the wire, while cannot detect any change to LSA made by the routers. This is because the OSPF packets are re-assembled on each hop, which is slightly different from IS-IS. So the problem for OSPF is mainly due to the problems inside the router, for example protocol implementations, system timers, or some hardware problem. Actually this problem has been seen in several production networks.

We can improve the description in the draft to make this clear.

Best regards,
Jie

From: Les Ginsberg (ginsberg) [mailto:[email-masked]]
Sent: Monday, August 01, 2016 1:30 PM
To: Dongjie (Jimmy); [email-masked]<mailto:[email-masked]>
Cc: Zhangxudong (zhangxudong, VRP); [email-masked]<mailto:[email-masked]>
Subject: RE: [OSPF] Solicit feedbacks on draft-dong-ospf-maxage-flush-problem-statement

Jie -

The draft says (Section 2):

"Since cryptographic authentication is executed at the OSPF packet
   level, it can only protect the assembled LSU packet for one hop and
   does not provide any additional protection for the corruption of LS
   age field."

But as authentication is calculated at the OSPF packet level, any change to the LS age field for an individual LSA contained within the OSPF packet (e.g. by some packet corruption in transmission) would cause authentication to fail when the packet is received. So the statement you make is not correct. I therefore am struggling to understand what problem you believe is not addressed by existing authentication techniques.

   Les

From: OSPF [mailto:[email-masked]] On Behalf Of Dongjie (Jimmy)
Sent: Sunday, July 31, 2016 8:15 PM
To: [email-masked]<mailto:[email-masked]>
Cc: Zhangxudong (zhangxudong, VRP); [email-masked]<mailto:[email-masked]>
Subject: [OSPF] Solicit feedbacks on draft-dong-ospf-maxage-flush-problem-statement

Hi all,

draft-dong-ospf-maxage-flush-problem-statement describes the problems caused by the corruption of the LS Age field, and summarizes the requirements on potential solutions. This draft received good comments during the presentation on the IETF meeting in B.A.

The authors would like to solicit further feedbacks from the mailing list, on both the problem statement and the solution requirements. Based on the feedbacks, we will update the problem statement draft, and work together to build suitable solutions.

The URL of the draft is:
https://tools.ietf.org/html/draft-dong-ospf-maxage-flush-problem-statement-00

Comments & feedbacks are welcome.

Best regards,
Jie

---

From: "Dongjie (Jimmy)" <[email-masked]>
Date: Mon, 1 Aug 2016 08:43:47 +0000

Hi Les,

Please see inline with [Jie]:

From: Les Ginsberg (ginsberg) [mailto:[email-masked]]
Sent: Monday, August 01, 2016 3:09 PM
To: Dongjie (Jimmy); [email-masked]
Cc: Zhangxudong (zhangxudong, VRP); [email-masked]
Subject: RE: [OSPF] Solicit feedbacks on draft-dong-ospf-maxage-flush-problem-statement

Jie -

[Jie]: Yes, for OSPF the corruption during packet transmission can be detected.

[Jie]: More specifically, the problem could be caused by either "setting the LS age field incorrectly due to implementation bug" or "system timer runs so fast that the LS age reaches MaxAge much earlier than other routers". Another less likely case is that the LS age field is corrupted before the LSA is assembled into OSPF packet.

[Jie]: Regarding the solutions space, IMO we need to consider both cases: "LS age reaches MaxAge" and "LS age close to MaxAge". For IS-IS, RFC 6232 and RFC 6233 provide solutions for the detection and identification of corrupted IS-IS purge, while OSPF does not have similar mechanisms.

This isn't correct.

[Jie] Thanks for pointing out this. This sentence need to be revised to mention "LSA corruption" rather than "packet corruption".

[Jie] Thanks for your suggestion. Yes we can update this draft with more specific problem statements as I mentioned above.

[Jie] As for the proposed solutions, the current draft specifies the requirements on the potential solutions, from which we envision that different solutions maybe needed for "Impact Mitigation" and "Problem Localization". Impact mitigation can be the easier one, for which we can start to discuss the potential solution. While the solution for "problem localization" may need more considerations.

Best regards,
Jie

   Les

To: Les Ginsberg (ginsberg); [email-masked]<mailto:[email-masked]>

Hi Les,

Best regards,
Jie

Jie -

   Les

Hi all,

The URL of the draft is:

Best regards,
Jie

---

From: "Les Ginsberg (ginsberg)" <[email-masked]>
Date: Mon, 1 Aug 2016 13:56:59 +0000

Jie -

Sent: Monday, August 01, 2016 1:44 AM

Hi Les,

Jie -

[Les:] It is incorrect to say that RFC 6232 makes it possible to detect a corrupt purge. What it does do is to provide an indication as to which IS initiated a purge. I don't know how OSPF would address this issue, but for OSPFv2 at least any solution would likely not be backwards compatible. For this reason I suggest that you not try to address this issue in the same draft.

Solutions to LS age  corruption can be done in a backwards compatible way, but they  MUST NOT result in discarding purges which pass authentication- doing so places you at risk for having inconsistent LSDBs in the network.

This isn't correct.

[Les:] A discussion of the requirements is useful and necessary, but IMO until you propose a solution there isn't enough substance for the document to become a WG document.

    Les

Best regards,
Jie

   Les

Hi Les,

Best regards,
Jie

Jie -

   Les

Hi all,

The URL of the draft is:

Best regards,
Jie

---

From: "Dongjie (Jimmy)" <[email-masked]>
Date: Tue, 2 Aug 2016 04:43:29 +0000

Hi Les,

Please see my replies with [Jie2]:

Sent: Monday, August 01, 2016 9:57 PM

Jie -

Hi Les,

Jie -

[Jie2]: Agreed, RFC 6232 provide the mechanism to track the misbehaved routers so that operator can fix the problem, the detection can be based on the rules in RFC 6233 or some other anomalies. Indeed for OSPFv2 legacy LSAs, it is difficult to introduce the mechanism similar to RFC 6232, while it can be easier for the OSPFv2/v3 Extended LSAs. So it depends on how backward compatible the solution should be. I agree with you that the solution for Problem Localization in OSPF needs to be provided in a separate document.

[Jie2]: Exactly. The received MaxAge LSAs cannot simply be discarded, the decision must be made carefully, probably based on some additional information. The authors has discussed some possible solution internally, and will prepare some material for further open discussion.

This isn't correct.

[Jie] As for the proposed solutions, the current draft specifies the requirements on the potential solutions, from which we envision that different solutions maybe needed for "Impact Mitigation" and "Problem Localization". The solution for "Impact mitigation" can be the easier one, for which we can start to discuss the potential solutions now. While the solution for "problem localization" may need more considerations.

[Jie2] Yes the current draft focuses on the problem statement and the requirements, the goal is to firstly get the MaxAge flush problem acknowledged and reach consensus on the requirements. Then the plan is to specify the solutions in separate documents.  Your valuable suggestions will be considered, and further contributions are welcome.

Best regards,
Jie

    Les

Best regards,
Jie

   Les

Hi Les,

Best regards,
Jie

Jie -

   Les

Hi all,

The URL of the draft is:

Best regards,
Jie

---

From: "Les Ginsberg (ginsberg)" <[email-masked]>
Date: Sun, 7 Aug 2016 18:13:46 +0000

Jie -

Thinking about the following some more:

<snip>

<end snip>

The benefits are extremely limited. If a router prematurely ages an LSA due to a timer bug, ignoring the received LSA age on reception isn't going to prevent premature purging by the router which has the bug. So the effect of ignoring the received LSA age prior to reaching MAXAGE will be short lived. You are then left with the possibility that an implementation corrupts the LSA age BEFORE calculating checksum/crypto authentication - but its local timeout logic is unaffected. This has very limited value. Whether the WG considers this worth pursuing is something you need to ask. For myself, I don't see much ROI here.

  Les

Sent: Monday, August 01, 2016 9:43 PM

Hi Les,

Jie -

Hi Les,

Jie -

This isn't correct.

Best regards,
Jie

    Les

Best regards,
Jie

   Les

Hi Les,

Best regards,
Jie

Jie -

   Les

Hi all,

The URL of the draft is:

Best regards,
Jie

---

From: "Acee Lindem (acee)" <[email-masked]>
Date: Mon, 8 Aug 2016 12:33:47 +0000

Hi Jie, Les, 

Speaking as a WG contributor:

From:  OSPF <[email-masked]> on behalf of "Les Ginsberg (ginsberg)"
<[email-masked]>
Date:  Sunday, August 7, 2016 at 2:13 PM
To:  Jie Dong <[email-masked]>, OSPF WG List <[email-masked]>
Cc:  "Zhangxudong (zhangxudong, VRP)" <[email-masked]>,
"[email-masked]" <[email-masked]>
Subject:  Re: [OSPF] Solicit feedbacks on
draft-dong-ospf-maxage-flush-problem-statement

>Jie –
>What remains is the possibility that an implementation has some bug and
>unintentionally modifies the age to something other than what it should
>be due to the actual elapsed time since LSA generation.
> I suppose a mechanism equivalent to what the IS-IS draft defined i.e.
>setting the age to “new” (0 in OSPF case) when first receiving a
>non-self-generated LSA could be useful to prevent negative impacts of
>such an implementation bug. Is this what you intend?
> 
>[Jie]: More specifically, the problem could be caused by either “setting
>the LS age field incorrectly due to implementation bug” or “system timer
>runs so fast that the LS age reaches MaxAge much
> earlier than other routers”. Another less likely case is that the LS age
>field is corrupted before the LSA is assembled into OSPF packet.
><end snip>
> 
>The benefits are extremely limited. If a router prematurely ages an LSA
>due to a timer bug, ignoring the received LSA age on reception isn’t
>going to prevent premature purging by the router which
> has the bug. So the effect of ignoring the received LSA age prior to
>reaching MAXAGE will be short lived. You are then left with the
>possibility that an implementation corrupts the LSA age BEFORE
>calculating checksum/crypto authentication – but its local timeout
> logic is unaffected. This has very limited value. Whether the WG
>considers this worth pursuing is something you need to ask. For myself, I
>don’t see much ROI here.
>
>
>

I agree with Les. Given the fact that a malicious router can wreck all
kinds of havoc in an OSPF routing domain anyway, the problem is limited to
cases where an OSPF router doesn’t adhere to the specification due to some
software bug. The case of packet corruption can be prevented by using
authentication which does include the LSA field in the message digest
computation. I think it is a slippery slope trying to fix bugs with
protocol enhancements.

Furthermore, I think possible solutions could be worse than the potential
problems themselves (due to complexity and the interaction with other OSPF
features such as DoNotAge LSAs). Envisioning solutions, I’d be opposed to
any protocol enhancements other than a fully backward compatible
enhancement to make the errant OSPF router easier to identify.

Thanks,
Acee 

>Subject: RE: [OSPF] Solicit feedbacks on
>draft-dong-ospf-maxage-flush-problem-statement
>
>
> 
>Hi Les,
>
> 
>From:
> Les Ginsberg (ginsberg) [mailto:[email-masked]]
>
>Cc: Zhangxudong (zhangxudong, VRP);
>[email-masked] <mailto:[email-masked]>
>Subject: RE: [OSPF] Solicit feedbacks on
>draft-dong-ospf-maxage-flush-problem-statement
>
>
> 
>Jie -
> 
>From:
> Dongjie (Jimmy) [mailto:[email-masked]]
>
>Cc: Zhangxudong (zhangxudong, VRP);
>[email-masked] <mailto:[email-masked]>
>Subject: RE: [OSPF] Solicit feedbacks on
>draft-dong-ospf-maxage-flush-problem-statement
>
>
> 
>Hi Les,
> 
>From:
> Les Ginsberg (ginsberg) [mailto:[email-masked]]
>
>Cc: Zhangxudong (zhangxudong, VRP);
>[email-masked] <mailto:[email-masked]>
>Subject: RE: [OSPF] Solicit feedbacks on
>draft-dong-ospf-maxage-flush-problem-statement
>
>
> 
>Jie –
> 
>https://www.ietf.org/id/draft-ietf-isis-remaining-lifetime-01.txt
>addresses problems
> where corruption of the remaining lifetime occurs either during
>transmission/reception or due to some DOS attack. This isn’t a concern w
>OSPF (hope you agree).
> 
>What remains is the possibility that an implementation has some bug and
>unintentionally modifies the age to something other than what it should
>be due to the actual
> elapsed time since LSA generation. I suppose a mechanism equivalent to
>what the IS-IS draft defined i.e. setting the age to “new” (0 in OSPF
>case) when first receiving a non-self-generated LSA could be useful to
>prevent negative impacts of such an implementation
> bug. Is this what you intend?
> 
>[Jie]: More specifically, the problem could be caused by either “setting
>the LS age field incorrectly due to implementation bug” or “system timer
>runs so fast that the LS age reaches
> MaxAge much earlier than other routers”. Another less likely case is
>that the LS age field is corrupted before the LSA is assembled into OSPF
>packet.
> 
>[Jie]: Regarding the solutions space, IMO we need to consider both cases:
>“LS age reaches MaxAge” and “LS age close to MaxAge”. For IS-IS, RFC 6232
>and RFC 6233 provide solutions for
> the detection and identification of corrupted IS-IS purge, while OSPF
>does not have similar mechanisms.
> 
>[Les:] It is incorrect to say that RFC 6232 makes it possible to detect a
>corrupt purge. What it does do is to provide an indication as to which IS
>initiated
> a purge. I don’t know how OSPF would address this issue, but for OSPFv2
>at least any solution would likely not be backwards compatible. For this
>reason I suggest that you not try to address this issue in the same draft.
> 
>[Jie2]: Agreed, RFC 6232 provide the mechanism to track the misbehaved
>routers so that operator can fix the problem, the detection can be based
>on the rules in RFC 6233 or some other
> anomalies. Indeed for OSPFv2 legacy LSAs, it is difficult to introduce
>the mechanism similar to RFC 6232, while it can be easier for the
>OSPFv2/v3 Extended LSAs. So it depends on how backward compatible the
>solution should be. I agree with you that the solution
> for Problem Localization in OSPF needs to be provided in a separate
>document.
> 
>Solutions to LS age  corruption can be done in a backwards compatible
>way, but they  MUST NOT result in discarding purges which pass
>authentication- doing so
> places you at risk for having inconsistent LSDBs in the network.
> 
>[Jie2]: Exactly. The received MaxAge LSAs cannot simply be discarded, the
>decision must be made carefully, probably based on some additional
>information. The authors has discussed some
> possible solution internally, and will prepare some material for further
>open discussion.
> 
>As written, the draft makes claims that are at least misleading – and I
>believe actually incorrect. In Section 6 you say:
> 
>“The LS age field may be altered as a result of
>   checksum nor OSPF packet cryptographic authentication.”
> 
>This isn’t correct.
> 
>[Jie] Thanks for pointing out this. This sentence need to be revised to
>mention “LSA corruption” rather than “packet corruption”.
> 
>What would be helpful – at least to me – is to move from a generic
>problem statement to the specific problem you want to solve and the
>proposed solution. This also
> requires you to more clearly state the cases where there is an actual
>vulnerability. It would be a lot easier to support the draft if this were
>done.
> 
>[Jie] Thanks for your suggestion. Yes we can update this draft with more
>specific problem statements as I mentioned above.
>
> 
>[Jie] As for the proposed solutions, the current draft specifies the
>requirements on the potential solutions, from which we envision that
>different solutions maybe needed for “Impact
> Mitigation” and “Problem Localization”. The solution for “Impact
>mitigation” can be the easier one, for which we can start to discuss the
>potential solutions now. While the solution for “problem localization”
>may need more considerations.
> 
>[Les:] A discussion of the requirements is useful and necessary, but IMO
>until you propose a solution there isn’t enough substance for the
>document to become
> a WG document.
> 
>[Jie2] Yes the current draft focuses on the problem statement and the
>requirements, the goal is to firstly get the MaxAge flush problem
>acknowledged and reach consensus
> on the requirements. Then the plan is to specify the solutions in
>separate documents.  Your valuable suggestions will be considered, and
>further contributions are welcome.
> 
>Best regards,
>Jie
> 
>    Les
> 
>Best regards,
>Jie
> 
>   Les
> 
> 
>From:
> Dongjie (Jimmy) [mailto:[email-masked]]
>
>Cc: Zhangxudong (zhangxudong, VRP);
>[email-masked] <mailto:[email-masked]>
>Subject: RE: [OSPF] Solicit feedbacks on
>draft-dong-ospf-maxage-flush-problem-statement
>
>
> 
>Hi Les,
>
> 
>OSPF packet level checksum and authentication can only protect the
>assembled LSU packet one hop on the wire, while cannot detect any change
>to LSA made by the routers. This is because
> the OSPF packets are re-assembled on each hop, which is slightly
>different from IS-IS. So the problem for OSPF is mainly due to the
>problems inside the router, for example protocol implementations, system
>timers, or some hardware problem. Actually this problem
> has been seen in several production networks.
> 
>From:
> Les Ginsberg (ginsberg) [mailto:[email-masked]]
>
>Cc: Zhangxudong (zhangxudong, VRP);
>[email-masked] <mailto:[email-masked]>
>Subject: RE: [OSPF] Solicit feedbacks on
>draft-dong-ospf-maxage-flush-problem-statement
>
>
> 
>Jie –
> 
>“Since cryptographic authentication is executed at the OSPF packet
>   age field.”
>But as authentication is calculated at the OSPF packet level, any change
>to the LS age field for an individual LSA contained within the OSPF
>packet (e.g. by some packet
> corruption in transmission) would cause authentication to fail when the
>packet is received. So the statement you make is not correct. I therefore
>am struggling to understand what problem you believe is not addressed by
>existing authentication techniques.
> 
>   Les
> 
> 
> 
>From:
> OSPF [mailto:[email-masked]]
>On Behalf Of Dongjie (Jimmy)
>To: [email-masked]
>Cc: Zhangxudong (zhangxudong, VRP);
>[email-masked] <mailto:[email-masked]>
>Subject: [OSPF] Solicit feedbacks on
>draft-dong-ospf-maxage-flush-problem-statement
>
>
> 
>Hi all,
> 
>draft-dong-ospf-maxage-flush-problem-statement describes the problems
>caused by the corruption of the LS Age field, and summarizes the
>requirements on potential solutions. This draft received good
> comments during the presentation on the IETF meeting in B.A.
> 
>The authors would like to solicit further feedbacks from the mailing
>list, on both the problem statement and the solution requirements. Based
>on the feedbacks, we will update the problem statement
> draft, and work together to build suitable solutions.
> 
>The URL of the draft is:

---

From: "Dongjie (Jimmy)" <[email-masked]>
Date: Thu, 11 Aug 2016 04:24:02 +0000

Hi Les,

The current draft is about problem statement, so IMO what the WG needs to consider is whether this is a vulnerability of OSPF protocol, and whether it can have negative impact to the network. If the problem is acknowledged, IMO it is worth to be documented.

The "ROI" as you mentioned is for the evaluation of the proposed solutions. I totally agree that for the timer bug case, recognizing and ignoring the received abnormal Maxage LSAs cannot stop the misbehaved router from generating further Maxage LSA, as it is a systematic problem, which can only be fixed after the operator identifies that router. This is also similar to the systematic corruption of IS-IS remain time.  And this is why this draft mentions two kinds of potential solutions, the mitigation mechanism can avoid the network being severely impacted by the problem, while for systematic problems, problem localization is needed to identify the misbehaved router and then solve the problem.

Best regards,
Jie

From: OSPF [mailto:[email-masked]] On Behalf Of Les Ginsberg (ginsberg)
Sent: Monday, August 08, 2016 2:14 AM
To: Dongjie (Jimmy) <[email-masked]>; [email-masked]
Cc: Zhangxudong (zhangxudong, VRP) <[email-masked]>; [email-masked]
Subject: Re: [OSPF] Solicit feedbacks on draft-dong-ospf-maxage-flush-problem-statement

Jie -

<snip>

  Les

Hi Les,

Jie -

Hi Les,

Jie -

This isn't correct.

Best regards,
Jie

    Les

Best regards,
Jie

   Les

Hi Les,

Best regards,
Jie

Jie -

   Les

Hi all,

The URL of the draft is:

Best regards,
Jie

---

From: "Les Ginsberg (ginsberg)" <[email-masked]>
Date: Thu, 11 Aug 2016 17:29:15 +0000

Jie -

Having the discussion has certainly been a good thing, but if the consensus of the WG is that there is no protocol change required then there is no need for any draft - which is my current position.

The other point is that you seem to be confusing the IS-IS Purge origination TLV (RFC 6232) with detecting invalid purges/remaining lifetime corruption. This is not the case. RFC 6232 simply allows us to detect which router originated a purge - it is not able to detect whether a purge is valid/invalid - and was not motivated by concerns about remaining lifetime corruption.

   Les

Sent: Wednesday, August 10, 2016 9:24 PM

Hi Les,

Best regards,
Jie

From: OSPF [mailto:[email-masked]] On Behalf Of Les Ginsberg (ginsberg)
Sent: Monday, August 08, 2016 2:14 AM
To: Dongjie (Jimmy) <[email-masked]<mailto:[email-masked]>>; [email-masked]<mailto:[email-masked]>
Cc: Zhangxudong (zhangxudong, VRP) <[email-masked]<mailto:[email-masked]>>; [email-masked]<mailto:[email-masked]>
Subject: Re: [OSPF] Solicit feedbacks on draft-dong-ospf-maxage-flush-problem-statement

Jie -

<snip>

  Les

Hi Les,

Jie -

Hi Les,

Jie -

This isn't correct.

Best regards,
Jie

    Les

Best regards,
Jie

   Les

Hi Les,

Best regards,
Jie

Jie -

   Les

Hi all,

The URL of the draft is:

Best regards,
Jie

---

From: "Acee Lindem (acee)" <[email-masked]>
Date: Fri, 12 Aug 2016 19:27:12 +0000

Speaking as a WG member:

Hi Jie,

I believe we agree that the problem is confined to OSPF bugs, system timer bugs,  and packet corruption. I’d assert that corruption can be detected via OSPF authentication. In fact, there is a well-known antidote where IS-IS authentication was enabled solely for the purpose of filtering corrupted protocol packets in an environment with line cards that were prone to such corruption. Hence, we are left with problems based on OSPF or system timer bugs. If there were a system timer bug, I’d doubt that networking device with such a bug would be functional to the point of being able to establish and maintaining OSPF adjacencies.  Do we really want to enhance the protocol to deal with bugs?

I’ve thought about this and one potential action I could envision would be to add a separate OSPF YANG notification where an LSA times out and a router other than the originator purges it. This way, the misbehaving OSPF router could be readily identified.

Thanks,
Acee

From: OSPF <[email-masked]<mailto:[email-masked]>> on behalf of "Les Ginsberg (ginsberg)" <[email-masked]<mailto:[email-masked]>>
Date: Thursday, August 11, 2016 at 1:29 PM
To: Jie Dong <[email-masked]<mailto:[email-masked]>>, OSPF WG List <[email-masked]<mailto:[email-masked]>>
Cc: "Zhangxudong (zhangxudong, VRP)" <[email-masked]<mailto:[email-masked]>>, "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>
Subject: Re: [OSPF] Solicit feedbacks on draft-dong-ospf-maxage-flush-problem-statement

Jie –

Having the discussion has certainly been a good thing, but if the consensus of the WG is that there is no protocol change required then there is no need for any draft – which is my current position.

The other point is that you seem to be confusing the IS-IS Purge origination TLV (RFC 6232) with detecting invalid purges/remaining lifetime corruption. This is not the case. RFC 6232 simply allows us to detect which router originated a purge – it is not able to detect whether a purge is valid/invalid – and was not motivated by concerns about remaining lifetime corruption.

   Les

Hi Les,

The “ROI” as you mentioned is for the evaluation of the proposed solutions. I totally agree that for the timer bug case, recognizing and ignoring the received abnormal Maxage LSAs cannot stop the misbehaved router from generating further Maxage LSA, as it is a systematic problem, which can only be fixed after the operator identifies that router. This is also similar to the systematic corruption of IS-IS remain time.  And this is why this draft mentions two kinds of potential solutions, the mitigation mechanism can avoid the network being severely impacted by the problem, while for systematic problems, problem localization is needed to identify the misbehaved router and then solve the problem.

Best regards,
Jie

Jie –

<snip>
What remains is the possibility that an implementation has some bug and unintentionally modifies the age to something other than what it should be due to the actual elapsed time since LSA generation. I suppose a mechanism equivalent to what the IS-IS draft defined i.e. setting the age to “new” (0 in OSPF case) when first receiving a non-self-generated LSA could be useful to prevent negative impacts of such an implementation bug. Is this what you intend?

[Jie]: More specifically, the problem could be caused by either “setting the LS age field incorrectly due to implementation bug” or “system timer runs so fast that the LS age reaches MaxAge much earlier than other routers”. Another less likely case is that the LS age field is corrupted before the LSA is assembled into OSPF packet.
<end snip>

The benefits are extremely limited. If a router prematurely ages an LSA due to a timer bug, ignoring the received LSA age on reception isn’t going to prevent premature purging by the router which has the bug. So the effect of ignoring the received LSA age prior to reaching MAXAGE will be short lived. You are then left with the possibility that an implementation corrupts the LSA age BEFORE calculating checksum/crypto authentication – but its local timeout logic is unaffected. This has very limited value. Whether the WG considers this worth pursuing is something you need to ask. For myself, I don’t see much ROI here.

  Les

Hi Les,

Jie -

Hi Les,

Jie –

https://www.ietf.org/id/draft-ietf-isis-remaining-lifetime-01.txt addresses problems where corruption of the remaining lifetime occurs either during transmission/reception or due to some DOS attack. This isn’t a concern w OSPF (hope you agree).

What remains is the possibility that an implementation has some bug and unintentionally modifies the age to something other than what it should be due to the actual elapsed time since LSA generation. I suppose a mechanism equivalent to what the IS-IS draft defined i.e. setting the age to “new” (0 in OSPF case) when first receiving a non-self-generated LSA could be useful to prevent negative impacts of such an implementation bug. Is this what you intend?

[Jie]: More specifically, the problem could be caused by either “setting the LS age field incorrectly due to implementation bug” or “system timer runs so fast that the LS age reaches MaxAge much earlier than other routers”. Another less likely case is that the LS age field is corrupted before the LSA is assembled into OSPF packet.

[Jie]: Regarding the solutions space, IMO we need to consider both cases: “LS age reaches MaxAge” and “LS age close to MaxAge”. For IS-IS, RFC 6232 and RFC 6233 provide solutions for the detection and identification of corrupted IS-IS purge, while OSPF does not have similar mechanisms.

[Les:] It is incorrect to say that RFC 6232 makes it possible to detect a corrupt purge. What it does do is to provide an indication as to which IS initiated a purge. I don’t know how OSPF would address this issue, but for OSPFv2 at least any solution would likely not be backwards compatible. For this reason I suggest that you not try to address this issue in the same draft.

As written, the draft makes claims that are at least misleading – and I believe actually incorrect. In Section 6 you say:

This isn’t correct.

[Jie] Thanks for pointing out this. This sentence need to be revised to mention “LSA corruption” rather than “packet corruption”.

What would be helpful – at least to me – is to move from a generic problem statement to the specific problem you want to solve and the proposed solution. This also requires you to more clearly state the cases where there is an actual vulnerability. It would be a lot easier to support the draft if this were done.

[Jie] As for the proposed solutions, the current draft specifies the requirements on the potential solutions, from which we envision that different solutions maybe needed for “Impact Mitigation” and “Problem Localization”. The solution for “Impact mitigation” can be the easier one, for which we can start to discuss the potential solutions now. While the solution for “problem localization” may need more considerations.

[Les:] A discussion of the requirements is useful and necessary, but IMO until you propose a solution there isn’t enough substance for the document to become a WG document.

Best regards,
Jie

    Les

Best regards,
Jie

   Les

Hi Les,

Best regards,
Jie

Jie –

   Les

Hi all,

The URL of the draft is:

Best regards,
Jie

---

From: "Dongjie (Jimmy)" <[email-masked]>
Date: Tue, 16 Aug 2016 13:16:29 +0000

Hi Les,

It seems that after these days discussion, now we are clear with the problem, this is a good progress for the problem statement draft. Next step I think we can focus on the discussion of the solutions.

If the Maxage flush problem happens in production network, without protocol change, it will have severe impact to the network, which I believe is not acceptable. At least some mechanism to mitigate the impact is needed. The mechanism you proposed can be part of the mitigation solution, while some optimization needs to be considered to avoid unexpected behaviors, e.g. in some cases the LSA may stay for quite a long time and cannot get aged properly.

As for RFC 6232, sorry for not making it clear in the beginning. RFC 6232 can be useful to track the originator of the purge message when an invalid purge (e.g. remain lifetime corruption) is detected by some other means.

Best regards,
Jie

Sent: Friday, August 12, 2016 1:29 AM

Jie -

   Les

Hi Les,

Best regards,
Jie

Jie -

<snip>

  Les

Hi Les,

Jie -

Hi Les,

Jie -

This isn't correct.

Best regards,
Jie

    Les

Best regards,
Jie

   Les

Hi Les,

Best regards,
Jie

Jie -

   Les

Hi all,

The URL of the draft is:

Best regards,
Jie

---

From: "Dongjie (Jimmy)" <[email-masked]>
Date: Tue, 16 Aug 2016 13:56:22 +0000

Hi Acee,

Thanks a lot for your feedbacks.

For packet corruption which impacts the LS age before the LSAs are packed into LSU packet, I agree it is less likely to happen than the other cases. However I think we agree that OSPF authentication only protect the packet level corruption, which cannot help to detect the corruption at LSA level.

In my understanding, robustness is an important feature of network protocols, which include the robustness to errors and failures happened in the network. If there is a bug in a particular router in the network, operator would not allow the whole network being impacted, which means other routers in the network needs to work properly in this situation. For example in BGP, the error handling mechanism has been optimized to avoid unnecessary session teardown.

I agree that OSPF Yang notification for LSA timeout is a nice thing to have and could be useful to identify the misbehaved router. My concern is sometimes the network may be severely impacted that the connectivity of netconf/restconf is also impacted. To avoid this, some mechanism to mitigate the impact of this problem could help.

Best regards,
Jie

From: Acee Lindem (acee) [mailto:[email-masked]]
Sent: Saturday, August 13, 2016 3:27 AM
To: Les Ginsberg (ginsberg); Dongjie (Jimmy); [email-masked]
Cc: Zhangxudong (zhangxudong, VRP); [email-masked]
Subject: Re: [OSPF] Solicit feedbacks on draft-dong-ospf-maxage-flush-problem-statement

Speaking as a WG member:

Hi Jie,

Thanks,
Acee

Jie –

   Les

Hi Les,

Best regards,
Jie

Jie –

<snip>

  Les

Hi Les,

Jie -

Hi Les,

Jie –

This isn’t correct.

Best regards,
Jie

    Les

Best regards,
Jie

   Les

Hi Les,

Best regards,
Jie

Jie –

   Les

Hi all,

The URL of the draft is:

Best regards,
Jie

---

From: "Les Ginsberg (ginsberg)" <[email-masked]>
Date: Tue, 16 Aug 2016 23:04:38 +0000

Jie -

I agree that the discussion has been useful. But my position - considering all the points made during this discussion - is that no protocol changes are advisable.
I therefore think the draft should not be moved forward.

   Les

Sent: Tuesday, August 16, 2016 6:16 AM

Hi Les,

Best regards,
Jie

Jie -

   Les

Hi Les,

Best regards,
Jie

Jie -

<snip>

  Les

Hi Les,

Jie -

Hi Les,

Jie -

This isn't correct.

Best regards,
Jie

    Les

Best regards,
Jie

   Les

Hi Les,

Best regards,
Jie

Jie -

   Les

Hi all,

The URL of the draft is:

Best regards,
Jie

---

From: "Dongjie (Jimmy)" <[email-masked]>
Date: Wed, 17 Aug 2016 09:44:33 +0000

Hi Les,

I'm a little confused here. Do you mean the problem statement should not be moved forward, because the discussion on the potential solutions has not converged yet?

We will prepare some material to describe a candidate solution for impact mitigation, and we can discuss further base on that . But IMO the progress of the solution does not impact the value of the problem statement, what do you think?

Best regards,
Jie

Sent: Wednesday, August 17, 2016 7:05 AM

Jie -

   Les

Hi Les,

Best regards,
Jie

Jie -

   Les

Hi Les,

Best regards,
Jie

Jie -

<snip>

  Les

Hi Les,

Jie -

Hi Les,

Jie -

This isn't correct.

Best regards,
Jie

    Les

Best regards,
Jie

   Les

Hi Les,

Best regards,
Jie

Jie -

   Les

Hi all,

The URL of the draft is:

Best regards,
Jie

---

From: "Les Ginsberg (ginsberg)" <[email-masked]>
Date: Wed, 17 Aug 2016 17:12:33 +0000

Jie -

My opinion is:

Problem statement draft should NOT move forward. There is no problem which we intend to solve.

   Les

Sent: Wednesday, August 17, 2016 2:45 AM

Hi Les,

Best regards,
Jie

Jie -

   Les

Hi Les,

Best regards,
Jie

Jie -

   Les

Hi Les,

Best regards,
Jie

Jie -

<snip>

  Les

Hi Les,

Jie -

Hi Les,

Jie -

This isn't correct.

Best regards,
Jie

    Les

Best regards,
Jie

   Les

Hi Les,

Best regards,
Jie

Jie -

   Les

Hi all,

The URL of the draft is:

Best regards,
Jie

---

From: "Acee Lindem (acee)" <[email-masked]>
Date: Wed, 17 Aug 2016 18:23:13 +0000

Speaking as a WG member who has some experience with OSPF implementations:

Hi Jie,

Along with Les, I’m also against progressing this draft.

From: Jie Dong <[email-masked]<mailto:[email-masked]>>
Date: Tuesday, August 16, 2016 at 9:56 AM
To: Acee Lindem <[email-masked]<mailto:[email-masked]>>, "Les Ginsberg (ginsberg)" <[email-masked]<mailto:[email-masked]>>, OSPF WG List <[email-masked]<mailto:[email-masked]>>
Cc: "Zhangxudong (zhangxudong, VRP)" <[email-masked]<mailto:[email-masked]>>, "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>
Subject: RE: [OSPF] Solicit feedbacks on draft-dong-ospf-maxage-flush-problem-statement

Hi Acee,

So, you are suggesting that LSAs are corrupted in the database in such a way that the LSA Age is set exactly to 0xE10? How would the implementation know that this had happened and prematurely age the packet? Database aging just doesn’t work this way (unless the implementation is particularly naïve).

So you agree your problem statement is confined to a software bug resulting in LSAs being aged too quickly? I think this is the third time I’ve raised this question.

If it has such a problem (whether it be due to a system timer bug or a some more specific aging problem), it seems the router would also be refreshing its LSAs all too frequently (at least at twice the rate) and it would be readily identifiable. For a system time problem, the router would likely have many other problems. For example, it would not maintain OSPF adjacencies if the dead timer advances fast enough. It would retransmit at a very fast rate as well. Are you going to write problem statements and suggest solutions for these situations as well?

What about other bugs? What if the router erroneously specifies a neighbor’s router-id as its own in a Router-LSA? Is this a problem the protocol should handle?

I believe a router have such impact would be easy to identify…

Thanks,
Acee

Best regards,
Jie

To: Les Ginsberg (ginsberg); Dongjie (Jimmy); [email-masked]<mailto:[email-masked]>

Speaking as a WG member:

Hi Jie,

Thanks,
Acee

Jie –

   Les

Hi Les,

Best regards,
Jie

Jie –

<snip>

  Les

Hi Les,

Jie -

Hi Les,

Jie –

This isn’t correct.

Best regards,
Jie

    Les

Best regards,
Jie

   Les

Hi Les,

Best regards,
Jie

Jie –

   Les

Hi all,

The URL of the draft is:

Best regards,
Jie

---

From: "Dongjie (Jimmy)" <[email-masked]>
Date: Thu, 18 Aug 2016 09:09:54 +0000

Hi Acee,

Please see my replies inline:

From: Acee Lindem (acee) [mailto:[email-masked]]
Sent: Thursday, August 18, 2016 2:23 AM
To: Dongjie (Jimmy); Les Ginsberg (ginsberg); [email-masked]
Cc: Zhangxudong (zhangxudong, VRP); [email-masked]
Subject: Re: [OSPF] Solicit feedbacks on draft-dong-ospf-maxage-flush-problem-statement

Hi Jie,

Hi Acee,

[Jie] Actually the case is when the LSA is about to be exchanged with neighbor, during the message packing the LS age is corrupted to either Maxage or a large number close to Maxage. The sending router does not intend to do a Maxage flush, however the neighbor routers which receive the message would treat this as a flush. This is a possible case although less likely to happen than the other cases.

[Jie] As I said before, the problems happened in the production network are caused by software bug in LSA aging, so I think this is the major case.

[Jie] This depends on the implementation. the software bug may only impact the aging of LSAs received from other routers. And frequent LSA refreshing may be caused by other cases such as link oscillation.  For a system timer problem, OSPF adjacency may oscillate, but if the management connection is impacted, such oscillation is difficult to be identified.

[Jie] Depends on the significance to network, case by case analysis may be needed.

[Jie] According to the feedback from on-site engineers, when IGP routing is oscillating severely which makes the management connection unavailable, it usually takes much longer time for troubleshooting, as logging to any router cannot be done via the management network. So maybe it would be better to have some automatic mechanism to reduce the impact before it becomes a big problem to troubleshoot.

Best regards,
Jie

Thanks,
Acee

Best regards,
Jie

Speaking as a WG member:

Hi Jie,

Thanks,
Acee

Jie –

   Les

Hi Les,

Best regards,
Jie

Jie –

<snip>

  Les

Hi Les,

Jie -

Hi Les,

Jie –

This isn’t correct.

Best regards,
Jie

    Les

Best regards,
Jie

   Les

Hi Les,

Best regards,
Jie

Jie –

   Les

Hi all,

The URL of the draft is:

Best regards,
Jie

---

From: "[email-masked]" <[email-masked]>
Date: Thu, 25 Aug 2016 02:11:45 +0800

Hello Jie, Acee and Les,

I am a coauthor of this draft from operator China Mobile. Thank you all for your discussion and suggestion in the previous mails. As you all discussed, a misbehavior OSPF router (due to software or hardware problem) can cause severe problem in the whole OSPF domain. 

Here I want to point out that OSPF route flapping DID occour in my field network contributed by a misbehavior OSPF router installed. The procedure to analyze and look for the cause were very complicated because we did not know the source of the flushing. Two hours past, we could not identify the real cause and restore our network. The CPU utilization of OSPF routers was high, the network traffic decreased significantly, lots of tunnel down warnings raised. When we tried to shutdown one OSPF router, route flapping stopped. This router was a newly deployed one. Through communication with our vendor, they admitted that this product had some defects in dealing with OSPF protocol. This kind of defects are difficult for us to test  when they apply for entrance in our network. Once defective products are deployed in the field network,  locating the problem is very hard and time consuming. 

So, I think it is necessary for us to solve the problem and improve the robustness of the protocol. At least it should provide the means to help us locate the OSPF route flapping problem.

[email-masked]
 
From: Dongjie (Jimmy)
Date: 2016-08-18 17:09
To: Acee Lindem (acee); Les Ginsberg (ginsberg); [email-masked]
CC: Zhangxudong (zhangxudong, VRP); [email-masked]
Subject: RE: [OSPF] Solicit feedbacks on draft-dong-ospf-maxage-flush-problem-statement
Hi Acee, 

 
 
Hi Jie, 

From: Jie Dong <[email-masked]>
Date: Tuesday, August 16, 2016 at 9:56 AM
To: Acee Lindem <[email-masked]>, "Les Ginsberg (ginsberg)" <[email-masked]>, OSPF WG List <[email-masked]>
Cc: "Zhangxudong (zhangxudong, VRP)" <[email-masked]>, "[email-masked]" <[email-masked]>
Subject: RE: [OSPF] Solicit feedbacks on draft-dong-ospf-maxage-flush-problem-statement
 
Hi Acee, 

 

 

 

 

Best regards,
Jie
 
Thanks,
Acee 

Best regards,
Jie

Speaking as a WG member: 
 
Hi Jie, 

 
Thanks,
Acee 

From: OSPF <[email-masked]> on behalf of "Les Ginsberg (ginsberg)" <[email-masked]>
Date: Thursday, August 11, 2016 at 1:29 PM
To: Jie Dong <[email-masked]>, OSPF WG List <[email-masked]>
Cc: "Zhangxudong (zhangxudong, VRP)" <[email-masked]>, "[email-masked]" <[email-masked]>
Subject: Re: [OSPF] Solicit feedbacks on draft-dong-ospf-maxage-flush-problem-statement
 
Jie –

 
   Les

Hi Les,

 
Best regards,
Jie

Jie –

<snip>

 
  Les

 
Hi Les, 

 
Jie -

Hi Les,

 
Jie –

 

 

 

 
 
This isn’t correct.

 

 
 
Best regards,
Jie
 
    Les
 
Best regards,
Jie
 
   Les

Hi Les, 

 
 
Best regards,
Jie

Jie –

 
 
   Les

 
Hi all,

 
The URL of the draft is:

Best regards,
Jie

---

From: "Les Ginsberg (ginsberg)" <[email-masked]>
Date: Wed, 24 Aug 2016 18:24:09 +0000

I completely agree that knowing the source of a purge is a valuable diagnostic tool. This has nothing to do with the corrupted maxage problem.

I think the conclusion thus far has been that we need no solution for the corrupted maxage.

It would be useful to have a way of determining the source of a purge – but in OSPF this is hard to do in a backwards compatible way. Acee has suggested using a management notification.
It would be useful to continue the discussion on this topic – but not the corrupt maxage issue.

   Les

From: [email-masked] [mailto:[email-masked]]
Sent: Wednesday, August 24, 2016 11:12 AM
To: Dongjie (Jimmy); Acee Lindem (acee); Les Ginsberg (ginsberg); [email-masked]
Cc: Zhangxudong (zhangxudong, VRP)
Subject: Re: RE: [OSPF] Solicit feedbacks on draft-dong-ospf-maxage-flush-problem-statement

Hello Jie, Acee and Les,

________________________________
[email-masked]<mailto:[email-masked]>

From: Dongjie (Jimmy)<mailto:[email-masked]>
Date: 2016-08-18 17:09
To: Acee Lindem (acee)<mailto:[email-masked]>; Les Ginsberg (ginsberg)<mailto:[email-masked]>; [email-masked]<mailto:[email-masked]>
CC: Zhangxudong (zhangxudong, VRP)<mailto:[email-masked]>; [email-masked]<mailto:[email-masked]>
Subject: RE: [OSPF] Solicit feedbacks on draft-dong-ospf-maxage-flush-problem-statement
Hi Acee,

To: Dongjie (Jimmy); Les Ginsberg (ginsberg); [email-masked]<mailto:[email-masked]>

Hi Jie,

Hi Acee,

Best regards,
Jie

Thanks,
Acee

Best regards,
Jie

Speaking as a WG member:

Hi Jie,

Thanks,
Acee

Jie –

   Les

Hi Les,

Best regards,
Jie

Jie –

<snip>

  Les

Hi Les,

Jie -

Hi Les,

Jie –

This isn’t correct.

Best regards,
Jie

    Les

Best regards,
Jie

   Les

Hi Les,

Best regards,
Jie

Jie –

   Les

Hi all,

The URL of the draft is:

Best regards,
Jie

---

From: "Dongjie (Jimmy)" <[email-masked]>
Date: Thu, 25 Aug 2016 07:53:19 +0000

Hi Les,

Yes knowing the source of purge/flush is quite useful when there is abnormal purge/flush issue in the network. The LSA flush caused by Maxage corruption is just one of these issues.  This problem may happen again as the LS age field is not protected by the LS checksum.

I agree that for OSPF it is difficult to find a totally backward compatible way to determine the source of flush, and the mechanism proposed by Acee can be one option.

IMO another equally useful thing is to avoid the severe disruption caused by abnormal LSA flushes before the source of flush is identified and fixed, and this mechanism can be designed in a backward compatible way, which would be something similar to the mechanism in draft-ietf-isis-remaining-lifetime.

Best regards,
Jie

From: Les Ginsberg (ginsberg) [mailto:[email-masked]]
Sent: Thursday, August 25, 2016 2:24 AM
To: [email-masked]; Dongjie (Jimmy) <[email-masked]>; Acee Lindem (acee) <[email-masked]>; [email-masked]
Cc: Zhangxudong (zhangxudong, VRP) <[email-masked]>
Subject: RE: RE: [OSPF] Solicit feedbacks on draft-dong-ospf-maxage-flush-problem-statement

   Les

From: [email-masked]<mailto:[email-masked]> [mailto:[email-masked]]
Sent: Wednesday, August 24, 2016 11:12 AM
To: Dongjie (Jimmy); Acee Lindem (acee); Les Ginsberg (ginsberg); [email-masked]<mailto:[email-masked]>
Cc: Zhangxudong (zhangxudong, VRP)
Subject: Re: RE: [OSPF] Solicit feedbacks on draft-dong-ospf-maxage-flush-problem-statement

Hello Jie, Acee and Les,

Hi Jie,

Hi Acee,

Best regards,
Jie

Thanks,
Acee

Best regards,
Jie

Speaking as a WG member:

Hi Jie,

Thanks,
Acee

Jie –

   Les

Hi Les,

Best regards,
Jie

Jie –

<snip>

  Les

Hi Les,

Jie -

Hi Les,

Jie –

This isn’t correct.

Best regards,
Jie

    Les

Best regards,
Jie

   Les

Hi Les,

Best regards,
Jie

Jie –

   Les

Hi all,

The URL of the draft is:

Best regards,
Jie

---

From: "[email-masked]" <[email-masked]>
Date: Fri, 26 Aug 2016 09:06:47 +0800

Hi Les,

Thank you very much for your comments and suggestion.

I will discuss with Jie Dong to submit a new version draft. This draft will illustrate the problem from the point of OSPF route flapping and corrupted maxage is one cause. After the discussion in the previous mails, we all think some methods should be proposed to solve this problem. The methods include both determining the source of flush and route flap mitigation if possible.

Best Regards,

From: Les Ginsberg (ginsberg)
Date: 2016-08-25 02:24
To: [email-masked]; Dongjie (Jimmy); Acee Lindem (acee); [email-masked]
CC: Zhangxudong (zhangxudong, VRP)
Subject: RE: RE: [OSPF] Solicit feedbacks on draft-dong-ospf-maxage-flush-problem-statement
I completely agree that knowing the source of a purge is a valuable diagnostic tool. This has nothing to do with the corrupted maxage problem.

 
   Les

Hello Jie, Acee and Les,

 

From: Dongjie (Jimmy)
Date: 2016-08-18 17:09

 
 
Hi Jie, 

 
Hi Acee, 

 

 

 

 

Best regards,
Jie
 
Thanks,
Acee 

Best regards,
Jie

Speaking as a WG member: 
 
Hi Jie, 

 
Thanks,
Acee 

Jie –

 
   Les

Hi Les,

 
Best regards,
Jie

Jie –

<snip>

 
  Les

 
Hi Les, 

 
Jie -

Hi Les,

 
Jie –

 

 

 

 
 
This isn’t correct.

 

 
 
Best regards,
Jie
 
    Les
 
Best regards,
Jie
 
   Les

Hi Les, 

 
 
Best regards,
Jie

Jie –

 
 
   Les

 
Hi all,

 
The URL of the draft is:

Best regards,
Jie