Subject: RE: Proposed Resolution to PROT I-D Issues List

From: "McDonald, Ira" <[email-masked]>
Date: Tue, 22 Mar 2005 05:27:59 -0800

Hi,

My apologies to Joel for misrepresenting our offline
discussion and his point of view.  I meant only to
summarize what Joel was trying to explain to me about
IESG policy.

[Juergen - I made that MIB ASN.1 argument offline
to Joel - and was told that the ASN.1 loses and
the plaintext wins - which I utterly disbelieve.
No MIB implementor ever ignored a SYNTAX or
MAX-ACCESS clause in favor of some plaintext.
And certainly no MIB compiler ever did this.]

But this list is still not addressing Hector's real
question, to whit:

Will the IESG allow the NetConf XSD to be used in
the protocol as a Normative _part_ of the incoming 
message validation logic?  

Joel, Randy, Wes, et al - could you please explain
to this list how XSD is useful in NetConf if it's
not Normative?

Cheers,
- Ira

Ira McDonald (Musician / Software Architect)
Blue Roof Music / High North Inc
PO Box 221  Grand Marais, MI  49839
phone: +1-906-494-2434
email: [email-masked]

-----Original Message-----
From: Joel M. Halpern [mailto:[email-masked]]
Sent: Monday, March 21, 2005 6:16 PM
To: McDonald, Ira; 'Wes Hardaker'; [email-masked]
Cc: 'Andy Bierman'; [email-masked]
Subject: RE: Proposed Resolution to PROT I-D Issues List

Just to be clear, I certainly never claimed to speak for the IESG.  I did 
describe IETF and IESG practice.

And I certainly never suggested that removing the XSD would in any way help 
the situation.

I objected privately to Ira's claim that making the English normative and 
the XSD informative was indefensible.  He seems to be taking objection to 
my attempting to illustrate other points of view which have a lot of 
history and are well tested in our community.

Yours,
Joel M. Halpern

At 06:06 PM 3/21/2005, McDonald, Ira wrote:
>Hi,
>
>I've been having a long, fruitless offline discussion with Joel
>Halpern about this.  The results may be summarized as follows:
>
>(1) The IESG policy is that the plaintext English is always
>     normative.
>
>(2) The IESG policy does NOT allow an appendix of XSD (or any
>     other relatively formal language) to override the plaintext
>     English, because then _parts_ of plaintext English sentences
>     or paragraphs might become invalid.
>
>I'm strongly opposed to this logic, because it implies that an RFC
>should not include any formal language appendices, since they're
>to be ignored anyway.
>
>Analogy - In English and US contract law, if an illegal or invalid
>statement is later found, then the clause or the subclause is
>invalid, but the rest of the contract remains legally valid.
>
>I suggest deleting all XSD from Appendices and references.  It
>won't do you any good anyway.  And changing the IESG policy
>is just hopeless.
>
>Discouraged,
>- Ira
>
>Ira McDonald (Musician / Software Architect)
>Blue Roof Music / High North Inc
>PO Box 221  Grand Marais, MI  49839
>phone: +1-906-494-2434
>email: [email-masked]
>
>-----Original Message-----
>From: [email-masked] [mailto:[email-masked]]On
>Behalf Of Wes Hardaker
>Sent: Monday, March 21, 2005 5:00 PM
>To: [email-masked]
>Cc: 'Wes Hardaker'; 'Andy Bierman'; [email-masked]
>Subject: Re: Proposed Resolution to PROT I-D Issues List
>
>
> >>>>> On Fri, 18 Mar 2005 09:03:01 -0800, "Steven Berl (sberl)"
><[email-masked]> said:
>
>Steven> Are you saying that we have a formal language description of
>Steven> the syntax of the protocol messages, but that is there just
>Steven> for information?
>
>My only intent was to state that it hasn't been proven to be perfect,
>and thus implementers should not rely on it as a check that an
>incoming packet is indeed perfect.
>
>The problem I've seen with XML applications is that many of them pass
>an incoming packet to a validator (which is merely validating the XML,
>not the data within) and then doesn't implement its own sanity error
>checking afterward.  Thus, I've actually seen many security problems
>in XML applications because they assume that the packet contains
>everything it needs in the exact form it needs when in fact it may
>not.
>
>Andy is right, however, that an XSD probably couldn't be written which
>meet every implementations requirements.
>
>If you're going to make it normative, I don't think it should go into
>the appendix as it is currently.  (If memory serves, appendices in
>RFCs are supposed to be normative).
>
>--
>"In the bathtub of history the truth is harder to hold than the soap,
>  and much more difficult to find."  -- Terry Pratchett
>
>--
>to unsubscribe send a message to [email-masked] with
>the word 'unsubscribe' in a single line as the message text body.
>archive: <http://ops.ietf.org/lists/netconf/>
>
>--
>to unsubscribe send a message to [email-masked] with
>the word 'unsubscribe' in a single line as the message text body.
>archive: <http://ops.ietf.org/lists/netconf/>

--
to unsubscribe send a message to [email-masked] with
the word 'unsubscribe' in a single line as the message text body.
archive: <http://ops.ietf.org/lists/netconf/>

---

From: Andy Bierman <[email-masked]>
Date: Tue, 22 Mar 2005 08:07:32 -0800

>And certainly no MIB compiler ever did this.]
MIB developers routinely interpret plaintext and SYNTAX together
in order to correctly implement a specification.   For example,
the MAX-ACCESS clause is not expressive enough to specify the
allowable write operation set for MIB tables which lock columns
if the row is active.  The sentence "This object may not be modified
if the associated fooRowStatus object is equal to 'active'" appears
in many DESCRIPTION clauses to get around this problem.

In the case of the NETCONF XSD, an implementor better understand how
capabilities affect the operation set that an agent accepts, the
transaction model it uses, etc., in order to do anything useful.

>to this list how XSD is useful in NetConf if it's
I don't think there is WG consensus to add the "informative only" text that
Wes proposed.  I think the "continuing WG consensus" is that we need an XSD
and namespace definition for the NETCONF protocol.  I don't know why we
need to add text to declare the XSD as normative or informative anyway.
This issue was already declared closed with "no changes required", and I 
don't
see any reason to change that now.

The XSD is a superset of all syntax allowed for the base plus standard 
capabilities. 
A real agent is not likely to support all the standard capabilities 
(e.g., agent will allow
writes to the <candidate> or to <running>, but not both).  So it's true 
that "if it's not
in the XSD, it's not in NETCONF", however no real agent will ever 
support the full XSD.

>- Ira
Andy

---

From: "Steven Berl \(sberl\)" <[email-masked]>
Date: Tue, 22 Mar 2005 08:46:59 -0800

I think what you are saying is that first a received message has to be valid
with respect to the XSD. Then, and only then, can further validity checks be
applied to determine whether the message can be accepted or not. 

Perhaps text something like this could be added:

"The XSD expresses a subset of the constraints that define a valid message.
If it doesn't pass that subset, then the message is not valid. The
particular subset of constraints expressed in the XSD are interesting
because the code to do the checking can be completely machine generated.
This doesn't mean that you don't need to do additional constraint checking
based on rules described in the text, the set of capabilities supported by
your implementation, and any other implementation imposed constraint rules."

If I could draw a Venn diagram of this in email I would. It would have 3
concentric circles that don't overlap at all. The outermost would be the set
of all well-formed XML messages. The next in would be the set of messages
that are valid with respect to the netconf XSD. The innermost would be the
subset of those valid messages that meet the additional constraints
expressed in the text. The edges of the circles never cross. Nothing outside
of the "valid against the XSD" circle can be part of the inner circle of
accepted messages.

Depending on the way the XSD is designed and developed it can express more
or less of the constraints. For example I could declare that an element is
just a xs:string, or I can add additional resrictions to say it must be a
string that matches a regular express. This changes the size of the "valid
against the XSD" circle, but it doesn't change the relationships described
in the paragraph above. In a perfect world, the "valid" circle and the
"accepted" circle would be the same, but because some of the constraints
can't be expressed in the XSD constraint language they are not. The "valid"
circle will always be bigger, and the rest of the rules will need to be
expressed in a different language, which in our case is english prose.

-steve

> From: [email-masked] 
> [mailto:[email-masked]] On Behalf Of Andy Bierman
> Sent: Tuesday, March 22, 2005 8:08 AM
> To: McDonald, Ira
> Cc: 'Joel M. Halpern'; 'Wes Hardaker'; [email-masked]; 
> [email-masked]
> discussion and his 
> >point of view.  I meant only to summarize what Joel was trying to 
> >explain to me about IESG policy.
> >
> >[Juergen - I made that MIB ASN.1 argument offline to Joel - and was 
> >told that the ASN.1 loses and the plaintext wins - which I utterly 
> >disbelieve.
> >No MIB implementor ever ignored a SYNTAX or MAX-ACCESS 
> clause in favor 
> >of some plaintext.
> allowable write operation set for MIB tables which lock 
> columns if the row is active.  The sentence "This object may 
> not be modified if the associated fooRowStatus object is 
> equal to 'active'" appears in many DESCRIPTION clauses to get 
> around this problem.
> 
> In the case of the NETCONF XSD, an implementor better 
> understand how capabilities affect the operation set that an 
> agent accepts, the transaction model it uses, etc., in order 
> to do anything useful.
> 
> >Will the IESG allow the NetConf XSD to be used in the protocol as a 
> >Normative _part_ of the incoming message validation logic?
> >
> >Joel, Randy, Wes, et al - could you please explain to this 
> list how XSD 
> >is useful in NetConf if it's not Normative?
> >  
> >
> I don't think there is WG consensus to add the "informative 
> only" text that Wes proposed.  I think the "continuing WG 
> consensus" is that we need an XSD and namespace definition 
> for the NETCONF protocol.  I don't know why we need to add 
> text to declare the XSD as normative or informative anyway.
> This issue was already declared closed with "no changes 
> required", and I don't see any reason to change that now.
> 
> The XSD is a superset of all syntax allowed for the base plus 
> standard capabilities. 
> A real agent is not likely to support all the standard 
> capabilities (e.g., agent will allow writes to the 
> <candidate> or to <running>, but not both).  So it's true 
> that "if it's not in the XSD, it's not in NETCONF", however 
> no real agent will ever support the full XSD.
> 
> >Cheers,
> >- Ira
> >  
> >
> Andy
> 
> >Ira McDonald (Musician / Software Architect) Blue Roof Music / High 
> >North Inc PO Box 221  Grand Marais, MI  49839
> >phone: +1-906-494-2434
> >Just to be clear, I certainly never claimed to speak for the 
> IESG.  I 
> >did describe IETF and IESG practice.
> >
> >And I certainly never suggested that removing the XSD would 
> in any way 
> >help the situation.
> >
> >I objected privately to Ira's claim that making the English 
> normative 
> >and the XSD informative was indefensible.  He seems to be taking 
> >objection to my attempting to illustrate other points of view which 
> >have a lot of history and are well tested in our community.
> >
> >Yours,
> >Joel M. Halpern
> >
> >>should not include any formal language appendices, since 
> they're to be 
> >>ignored anyway.
> >>
> >>statement is later found, then the clause or the subclause 
> is invalid, 
> >>but the rest of the contract remains legally valid.
> >>
> >>I suggest deleting all XSD from Appendices and references.  
> It won't 
> >>do you any good anyway.  And changing the IESG policy is just 
> >>hopeless.
> >>
> >>Discouraged,
> >>- Ira
> >>
> >>Ira McDonald (Musician / Software Architect) Blue Roof Music / High 
> >>North Inc PO Box 221  Grand Marais, MI  49839
> >>phone: +1-906-494-2434
> >>to unsubscribe send a message to 
> [email-masked] with the 
> >>word 'unsubscribe' in a single line as the message text body.
> >>to unsubscribe send a message to 
> [email-masked] with the 
> >>word 'unsubscribe' in a single line as the message text body.
> >to unsubscribe send a message to 
> [email-masked] with the 
> >word 'unsubscribe' in a single line as the message text body.
> to unsubscribe send a message to [email-masked] 
> with the word 'unsubscribe' in a single line as the message text body.

---

From: "Joel M. Halpern" <[email-masked]>
Date: Tue, 22 Mar 2005 12:11:22 -0500

The simplest answer is to look at other cases.
The IEEE 802.1d bridging spec was quite clear that the English was 
normative.  Nonetheless it include code for all the procedures, and that 
code was quite useful to people.

I think that it is more helpful to look at the question differently.
One expects even informative language in a specification to be 
helpful.  There is often a lot of informative language to help the reader 
understand the intent of the specification, and to help get effective 
results in the real world.

The only question under discussion is, if there is a disagreement between 
the English and the formal specification, which one should the reader 
assume was intended.   Such a disagreement is a bug in the spec, even if 
the part that is wrong is not normative.

In the absence of a disagreement, having the XSD will enable implementors 
to more reliably implement the intent of the specification.  This is helpful.

It can be argued that we should elevate it further.
We could go down the path of insisting that all normative behavior must be 
described either in the formal portions of the XSD or in comments 
explicitly part of the XSD.  That would make reading the specification and 
understanding it hard, but it would mean that we would be clear about what 
was intended to be binding.
We could say that the English is binding for semantics, but the XSD is 
binding for those things which it specifies.  This would make things very 
confusing.
There is admittedly also confusion in having the XSD but not making it 
normative.

Feel free to pick your confusion.
But please do not assert that having a differing viewpoint is nonsensical.

Yours,
Joel M. Halpern

---

From: "Joel M. Halpern" <[email-masked]>
Date: Tue, 22 Mar 2005 12:18:53 -0500

The other point is to echo Andy's comment.
Except in how we use captialized MUST, MAY, SHOULD wording, the IETF does 
not make a strong distinction in our specifications between normative and 
informative text.  (This is largely because we are not in the conformance 
testing business.)

As such, there is likely no reason to make an issue in the RFC of the 
question of what meaning applies in the event of conflict between text and 
the XSD.  There is a very good reason to make sure to the best of our 
ability that there is no such contradiction.

Yours,
Joel M. Halpern

---

From: "Randy Presuhn" <[email-masked]>
Date: Tue, 22 Mar 2005 10:35:37 -0800

Hi -

> From: "McDonald, Ira" <[email-masked]>
> To: "'Joel M. Halpern'" <[email-masked]>; "'Wes Hardaker'" <[email-masked]>; <[email-masked]>
> Cc: "'Andy Bierman'" <[email-masked]>; <[email-masked]>
> Sent: Tuesday, March 22, 2005 5:27 AM
...
> to this list how XSD is useful in NetConf if it's
...

It *is* normative.  It's just that in the case of conflict or ambiguity,
the English takes precedence.  This is as it should be.  I recall
fixing errors in ASN.1 and MIB modules, where the fix was justified
by the fact that the English text captured the WG intent and the
formal notation said something else.  This routinely happens during
MIB review cycles.

Randy

---

From: "Tom Petch" <[email-masked]>
Date: Tue, 22 Mar 2005 18:37:07 +0100

This issue surfaced just recently on another list and I am racking my brains to
remember where (I thought IPv6 but cannot see it there).  As I recall, it was a
discrepancy between ABNF and text with substantial disagreement as to which had
precedence; I think the ABNF won.

Tom Petch

----- Original Message -----
From: "Joel M. Halpern" <[email-masked]>
To: "McDonald, Ira" <[email-masked]>; "'Wes Hardaker'"
<[email-masked]>; <[email-masked]>
Cc: "'Andy Bierman'" <[email-masked]>; <[email-masked]>
Sent: Tuesday, March 22, 2005 6:18 PM
Subject: RE: Proposed Resolution to PROT I-D Issues List