Subject: Re: [Netconf] Consensus verification WAS:RE: AD review fordraft-ietf-netconf-tls-06.txt

From: "tom.petch" <[email-masked]>
Date: Sun, 15 Feb 2009 18:15:11 +0100

----- Original Message -----
From: "Ersue, Mehmet (NSN - DE/Munich)" <[email-masked]>
To: "Netconf" <[email-masked]>
Cc: <[email-masked]>
Sent: Friday, February 13, 2009 9:10 PM
Subject: [Netconf] Consensus verification WAS:RE: AD review
fordraft-ietf-netconf-tls-06.txt

> Dear NETCONF WG,
>
> we had already a long thread on this issue with 47 mails.
>
> As Bert stated in a parallel mail there seems to be no
> real practical or operational problems with the current
> end-of-message sequence. In fact changing the current
> mechanism would potentially cause problems.
>
> > >    Implementations MUST ensure that this character sequence is never
> > >    part of a NETCONF message.
>
> After some of you and the draft author agreed I assume
> we can see now the small text block above as the WG
> consensus.
>
> If anybody has any _strong_ objections please provide new
> text by Feb 16 by considering the discussion we had already.

I find this too terse, too coy.  Also, I am unclear how this fits w.r.t. the
current text or with the modification proposed at the start of this thread by
Dan.

I think we must be clearer.  There is no right answer as 47+ e-mails show.  I
agree that the protocol stays the same but we should point out the problem we
have now found, something like.

"[The] special character sequence, ]]>]]>, MUST be sent by both the client and
the server after each XML document in the NETCONF exchange, allowing
resynchronization of the NETCONF exchange in the event of an XML syntax or
parsing error.

"Contrary to the statement in RFC4742, the special character sequence can
legally appear in valid XML documents, as part of an attribute, of comments or
of Processing Instructions.  Implementations MUST ensure that this character
sequence is never part of a NETCONF message."

As to whether this should also be mentioned in s.5 of Security, I am inclined to
think that it should but that is only a _weak_ objection.

If someone reading this RFC sees this and thinks, 'Hey, they goofed' well, yes,
we goofed.  Better to say so.

Tom Petch

> Mehmet
>
>
> > -----Original Message-----
> > From: [email-masked]
> > [mailto:[email-masked]] On Behalf Of ext Mohamad Badra
> > Sent: Friday, February 13, 2009 2:24 AM
> > To: [email-masked]
> > Cc: Netconf
> > Subject: Re: [Netconf] AD review for draft-ietf-netconf-tls-06.txt
> >
> >
> > > On Thu, Feb 12, 2009 at 02:30:48PM +0100, Bert Wijnen (IETF) wrote:
> > >
> > >> I would like to ask for implementers and people who deploy
> > >> implementations
> > >> what the answer is to this question:
> > >>
> > >>   Has the current end-of-message sequence caused any
> > inter-operability
> > >>   problems, or just theoretical problems that only occur
> > in test suites?
> > >>
> > >> IF there are no real pragmatic/practical/operational
> > problems, then do
> > >> we
> > >> really need (want) to cover all possible/theoretical corner cases?
> > >
> > > Operationally, I am not so much concerned. I am more concerned about
> > > someone finding a way to make malicious use of this framing
> > sequence,
> > > e.g. tricking implementations to somehow generate framing sequences
> > > where they should not be and using that to do creative things.
> > >
> > > Some people on this list argue that such concerns are not justified
> > > because NETCONF implementations should never generate the framing
> > > sequence and if they do, the (not really defined) NETCONF access
> > > control prevents damage to happen and by tampering with
> > messages, you
> > > will not really be able to do damage but just cause parse errors and
> > > other failures.  My concern is that people might underestimate
> > > creativity of hackers and overestimate the skills of NETCONF
> > > implementors.
> > >
> > > But perhaps we are really best of for the moment stating explicitly
> > > that NETCONF implementations MUST ensure that the framing
> > sequence is
> > > never part of a NETCONF message they generate. By stating this
> > > explicitly this, a potential CERT advisory is immediately
> > turned away
> > > from the IETF and hits the implementors.
> > >
> > > So the errata becomes to replace the following text
> > >
> > >    This character sequence cannot
> > >    legally appear in an XML document, so it can be
> > unambiguously used to
> > >    identify the end of the current document, allowing
> > resynchronization
> > >    of the NETCONF exchange in the event of an XML syntax or parsing
> > >    error.
> > >
> > > with for example
> > >
> > >    Implementations MUST ensure that this character sequence is never
> > >    part of a NETCONF message.
> >
> >
> > I am fine with that.
> > Best regards,
> > Badra
> > _______________________________________________
> > Netconf mailing list
> > [email-masked]
> > https://www.ietf.org/mailman/listinfo/netconf
> >
> _______________________________________________
> Netconf mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/netconf

---

From: "Ersue, Mehmet (NSN - DE/Munich)" <[email-masked]>
Date: Mon, 16 Feb 2009 17:55:23 +0100

> sequence is never part of a NETCONF message."

I understand your intension, however having more text 
does not improve the clarity of the document. Do we really
need to state that NETCONF over TLS differs from RFC 4742? 

Let us try to find a consensus with following text:

"This document uses the same delimiter sequence ("]]>]]>") defined in 
[RFC4742], which MUST be sent by both the client and the server after 
each XML document in the NETCONF exchange. 
Implementations MUST ensure that this character sequence is never 
part of a NETCONF message. This character sequence can legally 
appear as carried data of XML elements in valid XML documents."

The consequence is that the TLS draft will have a different 
delimiter handling as current 4742. I suggest to bring in this 
as an errata for 4742.

Mehmet

---

From: "David B Harrington" <[email-masked]>
Date: Mon, 16 Feb 2009 16:23:53 -0500

Hi,

Well, Juergen started out describing my concerns that the framing
sequence problem could be deliberately misused. 

But I don't think the solution to develop a rule that says "don't do
this" will work that well. If I were a hacker, and saw this rule, that
would probably be one of the very places I would look for an attack
vector.

I think the WG needs to seriously consider how this could be misused.

The Security Considerations should probably discuss this. Could an
attacker use this to create a DoS attack? 

dbh

> [mailto:[email-masked]] On Behalf Of Juergen Schoenwaelder
> Sent: Thursday, February 12, 2009 5:10 PM
> To: Bert Wijnen (IETF)
> Cc: Netconf
> >   problems, or just theoretical problems that only occur in 
> test suites?
> >
> problems, then do we
sequence,
> control prevents damage to happen and by tampering with messages,
you
> that NETCONF implementations MUST ensure that the framing sequence
is
> explicitly this, a potential CERT advisory is immediately turned
away
> And someone might formulate some additional text explaining why this
> is not a restriction for any data carried in XML elements etc.
> 
> /js
> 
> -- 
> Juergen Schoenwaelder           Jacobs University Bremen gGmbH
> Phone: +49 421 200 3587         Campus Ring 1, 28759 Bremen, Germany
> Fax:   +49 421 200 3103         <http://www.jacobs-university.de/>

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Mon, 16 Feb 2009 22:37:25 +0100

> attacker use this to create a DoS attack? 

This is a classic code injection problem. An attacker might be able to
inject arbitrary NETCONF messages via some application that does not
carfully check user input. Hence, applications and NETCONF APIs better
ensure that framing sequence never show up in NETCONF messages.

I do not really see a DoS attack in the classic sense of the work. Of
course, if someone injects NETCONF messages to change lets say your
BGP config, this might have quite interesting consequences. We are
after all talking about configuration...

I agree that a paragraph or two in the Security Considerations is a
good thing as it might help implementors to understand that it is
really important to carefully check user provided input and the format
of NETCONF messages generated.

/js

---

From: "David B Harrington" <[email-masked]>
Date: Mon, 16 Feb 2009 16:39:01 -0500

Hi Mehmet,

"Implementations" is not that clear. Must all NETCONF implementations
prevent this? Or is it the implementations of NETCONF over TLS that
MUST handle this?  Does this sequence pose the same problems for other
transports, such as BEEP and SOAP and SSH? 

I think the text needs to be clearer.

What happens if NETCONF is processing an outgoing message and finds
this in the NETCONF message? what does it do - throw the message away?
escape the sequence? strip the sequence? Should that portion of the
processing return an error to the application that originated the
message? if so, which error code?

What happens if NETCONF is processing an incoming message and finds
this in the NETCONF message? what does it do - throw the message away?
Process the partial message? if so, what impact could that have on
network stability and/or network security? Is there any possibility
that a man-in-the-middle could insert such a sequence into a NETCONF
message? Do all the transports have mechanisms that prevent message
content modification, such that this should never be a problem?

dbh 

> [mailto:[email-masked]] On Behalf Of Ersue, Mehmet 
> (NSN - DE/Munich)
> Sent: Friday, February 13, 2009 3:11 PM
> To: Netconf
> Cc: [email-masked]

---

From: "Mohamad Badra" <[email-masked]>
Date: Tue, 17 Feb 2009 00:42:00 +0100 (CET)

Hi David

The man-in-the-middle is not possible between the two TLS entities (if the
identity check is well established as described in the document). However,
the transport layers can't resolve this problem if it happen at the upper
layers (inside attacks, out of TLS scope).

Since we must use the same RFC4742 special sequence, I think we can
alwaysuse it but in conjunction with another value generated by the TLS
session itself (let's say the TLS client random value, or the first 4
bytes of this value (the current gmt_unix_time)) or a value known only by
the TLS client and server.

If that is possible(I hope so :-)), so the TLS entities (client or server)
will replace the special sequence appearing in XML documents - exchanged
between two TLS entities - with the TLS session_id, add the special
sequence at the end of the modified XML document and send the result over
the TLS secure session (where date integrity is ensured). The TLS entity
that receives this modified document will look for the special sequence to
delimit the modified XML document (only one special sequence could appear
by a XML document). Next, the same entity will look for the TLS session_id
in the modified document to replace it with the special sequence before
delivering the original document to the application layer.

Well, it is always possible to say that a man in the middle can read the
TLS parameters exchanged in clear text (e.g. random values, session
identifier, etc.) and then can inject/insert falsified data, etc. IMO, we
should then use a secret sequence computed from the TLS material key to
avoid such a problem.

Comments?

Best regards,
Badra

---

From: WashamFan <[email-masked]>
Date: Tue, 17 Feb 2009 10:46:31 +0800

>  delivering the original document to the application layer.

Do you bring another problem? Is it possible that TLS session_id itself is 
legal in original XML document? If yes, you would modify the original 
document inadvertently.

washam

---

From: "Bert Wijnen \(IETF\)" <[email-masked]>
Date: Tue, 17 Feb 2009 14:03:00 +0100

David,

You have lost of questions. Pls realize:
- we are in IETF LC phase. It would be good to add suggested text
  to remedy the rpoblems you see.

- your questions equally apply to the rfc4742 context.

- So... do we want to have a NetConf over TLS that is in the same 
  good/reasnable shape as NetConf over SSH? Or do we want to
  put all the concerns we have on the generic EOM issue on
  this specific document?

- I would suggest that we try to get this document approved
  and published, so that people can start  to implement against
  a stable (as stable as rfc4742) spec.

- If we still see the sort of issues you bring forward as SERIOUS
  enough to tackle them, then we should probably launch a rfc4742
  and rfcyyyy (the Netconf over TLS spec once it is an RFC) update
  to fix any gaps/errors/security/hacking concerns.

I am not sure that the suggested extra TLS-generated value is gonna
get us anywhere in a reasonable amount of time. And even if it does,
it does not solve the same (generic?) problem with the EOM handling
in RFC4742.

Bert

----- Original Message ----- 
  From: Mohamad Badra 
  To: David B Harrington 
  Cc: 'Netconf' 
  Sent: Tuesday, February 17, 2009 12:42 AM
  Subject: Re: [Netconf] Consensus verification WAS:RE: AD reviewfordraft-ietf-netconf-tls-06.txt

  Hi David

  Comments?

  Best regards,
  Badra

---

From: Badra <[email-masked]>
Date: Tue, 17 Feb 2009 22:47:54 +0100

Dear all,

The Security Considerations should discuss the misused of the special
sequence.

Let us try to find a consensus with the following text:

   "An attacker might be able to inject arbitrary NETCONF messages via
   some application that does not carefully check NETCONF messages.
   Hence, applications and NETCONF APIs should ensure that the delimiter
   sequence defined in section 2.1 never appears in NETCONF messages.
   Implementations of this document should cause parse errors and other
   failures if they find this delimiter sequence in a NETCONF message."

Best regards,
Badra

On Tue, Feb 17, 2009 at 2:03 PM, Bert Wijnen (IETF) <[email-masked]>wrote:

> *From:* Mohamad Badra <[email-masked]>
> *To:* David B Harrington <[email-masked]>
> *Cc:* 'Netconf' <[email-masked]>
> *Sent:* Tuesday, February 17, 2009 12:42 AM
> *Subject:* Re: [Netconf] Consensus verification WAS:RE: AD
> reviewfordraft-ietf-netconf-tls-06.txt
>
> Hi David
>

---

From: Badra <[email-masked]>
Date: Tue, 17 Feb 2009 23:22:15 +0100

I forgot to send the portion of the text related to the delimiter (Section
2.1)

So please let us try to find a consensus For

1- Section 2.1: [This text differs with only one addition ("of this
document") from the mail sent by Mehmet on Mo 16.02.2009 17:55.]

   "This document uses the same delimiter sequence ("]]>]]>") defined in
   [RFC4742], which MUST be sent by both the client and the server after
   each XML document in the NETCONF exchange.
   Implementations of this document MUST ensure that this character
   sequence is never part of a NETCONF message. This character sequence
   can legally appear as carried data of XML elements in valid XML
documents."

2- Security Considerations:

Best regards,
Badra

-- 
Badra

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Wed, 18 Feb 2009 09:04:56 +0100

>    can legally appear as carried data of XML elements in valid XML

I have trouble with the last sentence because it uses the term "XML
element" and this is likely misleading. I think it deserves some
discussion (a) where this sequence may appear legally in a well-formed
XML document and (b) where it may not appear legally in a well-formed
XML document.
 
>    failures if they find this delimiter sequence in a NETCONF message."

I am not sure about the parse error since bogus injection happens on
the sending side, not on the receiving side (and some people argued
that the sequence is good for recovering synchronization - I
personally would prefer to close the session but that is just me).

So my suggestion is the remove the last sentence and turn the should
in the second sentence into a MUST.

/js

---

From: Badra <[email-masked]>
Date: Wed, 18 Feb 2009 09:27:54 +0100

> XML document and (b) where it may not appear legally in a well-formed

How about:

   This character sequence can legally appear as carried data of XML
   elements in valid XML documents, as part of an attribute, of
   comments orof Processing Instructions.

 Please don't hesitate to send another proposal if you disagree with the
above text.

> I am fine with that

Best regards
Badra

---

From: "tom.petch" <[email-masked]>
Date: Wed, 18 Feb 2009 09:37:28 +0100

----- Original Message -----
From: "Ersue, Mehmet (NSN - DE/Munich)" <[email-masked]>
To: "tom.petch" <[email-masked]>; "Netconf" <[email-masked]>
Cc: <[email-masked]>
Sent: Monday, February 16, 2009 5:55 PM
Subject: RE: [Netconf] Consensus verification WAS:RE: AD review

<tp>

Mmmmmmmmmmm

OK, in the interests of progress, I will compromise with that wording.

Also, I have no problem with the use of 'implementations'.

<aside>
Having more text now might save significant effort in a year or two's time,
in the same way that having more text from 2004 would have saved us time
in November and now, so my thinking on having more text diverges from yours.
Thus I now know that the sequence can appear in more than just 'XML elements'
which I might have forgotten when next this surfaces, but as I say, I will
compromise on that:-)
</aside>

Tom Petch

</tp>

Mehmet

---

From: Martin Bjorklund <[email-masked]>
Date: Wed, 18 Feb 2009 12:51:48 +0100 (CET)

>    comments orof Processing Instructions.

It cannot be part of XML element data (not even CDATA), which
apparently is why it was choosen.

    This character sequence can legally appear as part of XML
    attributes, of XML comments, and of Processing Instructions.

/martin

---

From: Badra <[email-masked]>
Date: Thu, 19 Feb 2009 19:45:55 +0100

Dear all,

I considered all concerns until now and prepared following text blocks for
the delimiter handling in section 2.1 and the security considerations
section.

We are going to use these text blocks now for the update of the document for
the next step.

Please send me your comments if you have any strong objections.
Best regards,
Badra

1) Section 2.1

   "This document uses the same delimiter sequence ("]]>]]>") defined in
   [RFC4742], which MUST be sent by both the client and the server after
   each XML document in the NETCONF exchange.  Since this character
   sequence can legally appear in plain XML in attribute values, comments,
   and Processing Instructions, implementations of this document MUST
   ensure that this character sequence is never part of a NETCONF message."

2) Security Considerations

   "The security considerations described throughout [RFC5246] and
   [RFC4741] apply here as well.

   This document in its current version doesn't support third party
   authentication due to the fact that TLS doesn't specify this way of
   authentication and that NETCONF depends on the transport protocol for
   the authentication service.  If third party authentication is needed,
   BEEP or SSH transport can be used.

   An attacker might be able to inject arbitrary NETCONF messages via some
   application that does not carefully check exchanged messages or
   deliberately insert the delimiter sequence in a Netconf message to
   create a DoS attack.  Hence, applications and NETCONF APIs must ensure
   that the delimiter sequence defined in section 2.1 never appears in
   NETCONF messages;  otherwise, those messages can be dropped, garbled or
   mis-interpreted. If it is found in an incoming NETCONF message by the
   sender side, a robust implementation of this document should warn the
   user that illegal characters have been discovered.  If the delimiter
   sequence is found in a Netconf message by the receiver side (including
   any XML attributes, XML comments or Processing Instructions) a robust
   implementation of this document must silently discard the message
   without further processing and then stop the NETCONF session.

   Finally, this document does not introduce any new security
   considerations compared to [RFC4742]."

---

From: "tom.petch" <[email-masked]>
Date: Fri, 20 Feb 2009 14:26:20 +0100

---- Original Message -----
From: "Badra" <[email-masked]>
To: "Martin Bjorklund" <[email-masked]>
Cc: <[email-masked]>
Sent: Thursday, February 19, 2009 7:45 PM
Subject: Re: [Netconf] Consensus verification WAS:RE:
ADreviewfordraft-ietf-netconf-tls-06.txt

>    considerations compared to [RFC4742]."

If we accept these semantics, I see a need for some editorial tweaks:

NETCONF or Netconf? don't mind but would like consistency

/NETCONF APIs must/NETCONF APIs MUST/

/If it is found/If the delimiter sequence is found/

'incoming message by the sender side' um, I think of incoming to receivers, not
senders, suggest dropping the word 'incoming'

Tom Petch