Subject: [netmod] Yang 1.0/1.1 ABNF Grammar: <a string that matches the rule ...>

From: Robert Wilton <[email-masked]>
Date: Tue, 13 Oct 2015 11:01:12 +0100

Hi,

I was looking the Yang 1.1 ABNF Grammar, and noticed various places 
where the rules are specified as the directive "a string that matches 
the rule XXX", e.g.

    yang-version-arg-str = < a string that matches the rule
                            yang-version-arg >

    yang-version-arg    = "1"

It was slightly unclear to me exactly what is meant by this.

Am I right in understanding that the only valid text that would match 
yang-version-arg-str would be the 3 character sequence *"1"*, or is it 
more nuanced that this?

I.e. I presume that there is a good reason why these rules aren't just 
specified as the following:

yang-version-arg-str = DQUOTE yang-version-arg DQUOTE

Thanks,
Rob

---

From: Jernej Tuljak <[email-masked]>
Date: Tue, 13 Oct 2015 12:07:12 +0200

> yang-version-arg-str = DQUOTE yang-version-arg DQUOTE

The reason are probably YANG quotation rules. The "string that matches" 
is a dequoted string without any "+" or quotes, so the grammar rules 
don't have to handle those explicitly.

Jernej

>
> Thanks,
> Rob
>
>
>
> _______________________________________________
> netmod mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/netmod

---

From: Martin Bjorklund <[email-masked]>
Date: Tue, 13 Oct 2015 12:17:49 +0200 (CEST)

Robert Wilton <[email-masked]> wrote:
> yang-version-arg-str would be the 3 character sequence *"1"*

No.

> or is it

Yes.

First of all the abnf-rule

  yang-version-arg = "1"

matches a single character 1 (not three characters).

Next, YANG has a very regular syntax that all statements follow:

     statement = keyword [argument] (";" / "{" *statement "}")

The argument in this rule is a string.  A string in YANG can be
specified in several ways:

   unqouted, e.g.:    foo
   quoted, e.g.:      'foo' or "foo"
   concatenated, e.g: "f" + "oo"

This cannot be expressed directly in ABNF.  So when the YANG grammar
says "a string that matches the rule..." it means that the "expanded"
string matches the rule.

So e.g., this is all legal:

    config false;
    config "false";
    config 'f' + "alse";

/martin

---

From: Robert Wilton <[email-masked]>
Date: Tue, 13 Oct 2015 11:17:38 +0100

Hi Jernej,

Thanks for the explanation, that along with looking at section 
6.1.2/6.1.3 makes this clear.

Cheers,
Rob

On 13/10/2015 11:07, Jernej Tuljak wrote:
>> yang-version-arg-str would be the 3 character sequence *"1"*, or is 
>> it more nuanced that this?
>>
>> I.e. I presume that there is a good reason why these rules aren't 
>> just specified as the following:
>>
> The reason are probably YANG quotation rules. The "string that 
> matches" is a dequoted string without any "+" or quotes, so the 
> grammar rules don't have to handle those explicitly.
>
> Jernej
>
>>
>> Thanks,
>> Rob
>>
>>
>>

---

From: Robert Wilton <[email-masked]>
Date: Wed, 14 Oct 2015 10:25:08 +0100

Hi Martin,

I was looking at the YANG ABNF grammar a bit more last night (to see how 
hard it would be to write a parser for it) and I had a couple more 
observations.  Apologies that this is after the WG last call ...

1. [Trivial] The indentation of the range statement in 9.3.5 looks wrong.

9.3.5. Usage Example

      typedef my-decimal {
        type decimal64 {
          fraction-digits 2;
            range "1 .. 3.14 | 10 | 20..max";
        }
      }

I presume that it should be:

9.3.5. Usage Example

      typedef my-decimal {
        type decimal64 {
          fraction-digits 2;
          range "1 .. 3.14 | 10 | 20..max";
        }
      }

2.  The description of yang-char (around page 186) doesn't seem to be 
quite accurate (relative to description of legal characters in 6. YANG 
Syntax), and given that it excludes character values outside the unicode 
range.

    ;; any Unicode character including tab, carriage return, and line
    ;; feed, but excluding the other C0 control characters, the surrogate
    ;; blocks, and the noncharacters.
    yang-char = %x9 / %xA / %xD / %x20-D7FF /
    ...

Should this be:

    ;; any Unicode or IOS/IEC 10646 character including tab, carriage return, and line
    ;; feed, but excluding the other C0 control characters, the surrogate
    ;; blocks, and the noncharacters.
    yang-char = %x9 / %xA / %xD / %x20-D7FF /

3. There are lots of comments where "these stmts can appear in any 
order", e.g.

    linkageStmts       = ;; these stmts can appear in any order
                          *importStmt
                          *includeStmt

Am I right in interpreting that there can be any number of import and 
include statements and they can be interleaved in any arbitrary order?  
E.g. this specific example (but not in the general case) could equally 
have been written *(importStmt / includeStmt).

Thanks,
Rob

On 13/10/2015 11:17, Martin Bjorklund wrote:
> .

---

From: Martin Bjorklund <[email-masked]>
Date: Wed, 14 Oct 2015 19:14:52 +0200 (CEST)

Robert Wilton <[email-masked]> wrote:
> Hi Martin,
> 
> I was looking at the YANG ABNF grammar a bit more last night (to see
> how hard it would be to write a parser for it) and I had a couple more

Fixed.

> Syntax), and given that it excludes character values outside the
> unicode range.

Hmm, which characters are outside the unicode range?

>    ;; any Unicode or IOS/IEC 10646 character including tab, carriage return,
>    ;; and line

I think this would be ok.

Yes.

> have been written *(importStmt / includeStmt).

Well, the grammar defines the canonical order.  With the alternative
rule above, the canonical order would be different.

/martin

---

From: Robert Wilton <[email-masked]>
Date: Wed, 14 Oct 2015 19:48:45 +0100

> Hmm, which characters are outside the unicode range?
I was thinking of anything above 0xFFFF, but it looks like my definition 
(and possibly quite a few others on the Internet) of Unicode vs UTF-16 
is out of date.

> rule above, the canonical order would be different.
Thanks for the clarification,
Rob

---

From: Robert Wilton <[email-masked]>
Date: Thu, 22 Oct 2015 13:27:53 +0100

Hi Martin,

I have a couple more minor queries/observations as I work through some 
of the details of ABNF grammar:

1. For the module-stmt rule below, to be strictly correct, should it 
have a comment stating that the statements in any of the "*-stmts" 
blocks can appear in any order?  Or is the intention that the "*-stmt" 
blocks must strictly be in the order defined by the ABNF? If the latter 
statement is true then does any of the text in 7.1 need to be 
strengthened to explicitly state this?

module-stmt         = optsep module-keyword sep identifier-arg-str
                          optsep
                          "{" stmtsep
                              module-header-stmts
                              linkage-stmts
                              meta-stmts
                              revision-stmts
                              body-stmts
                          "}" optsep

2. Similarly for import-stmt.  Should this have a comment indicating 
that prefix-stmt or revision-date-stmt can appear in any order?

    import-stmt         = import-keyword sep identifier-arg-str optsep
                          "{" stmtsep
                              prefix-stmt
                              [revision-date-stmt]
                          "}" stmtsep

On a practical note, it seems that YANG allowing various statements to 
be in any arbitrary order makes writing a parser quite a lot more 
complex and less efficient than writing a parser that only accepts YANG 
modules that have been written in the canonical order.

Thanks,
Rob

On 14/10/2015 19:48, Robert Wilton wrote:
> I was thinking of anything above 0xFFFF, but it looks like my 
> definition (and possibly quite a few others on the Internet) of 
> Unicode vs UTF-16 is out of date.
>
>>

---

From: Ladislav Lhotka <[email-masked]>
Date: Thu, 22 Oct 2015 14:50:45 +0200

> On 22 Oct 2015, at 14:27, Robert Wilton <[email-masked]> wrote:
> 
> Hi Martin,
> 
> I have a couple more minor queries/observations as I work through some of the details of ABNF grammar:
> 
> 1. For the module-stmt rule below, to be strictly correct, should it have a comment stating that the statements in any of the "*-stmts" blocks can appear in any order?  Or is the intention that the "*-stmt" blocks must strictly be in the order defined by the ABNF? If the latter statement is true then does any of the text in 7.1 need to be strengthened to explicitly state this?

I understand the order is fixed in this case.

> 2. Similarly for import-stmt.  Should this have a comment indicating that prefix-stmt or revision-date-stmt can appear in any order?
> 

Here it IMO makes little sense to require fixed order.

Lada

> 
> 
> On a practical note, it seems that YANG allowing various statements to be in any arbitrary order makes writing a parser quite a lot more complex and less efficient than writing a parser that only accepts YANG modules that have been written in the canonical order.
> 
> Thanks,
> Rob
> 
> 
>> I was thinking of anything above 0xFFFF, but it looks like my definition (and possibly quite a few others on the Internet) of Unicode vs UTF-16 is out of date.
>> 
>>> 

--
Ladislav Lhotka, CZ.NIC Labs
PGP Key ID: E74E8C0C

---

From: Robert Wilton <[email-masked]>
Date: Thu, 22 Oct 2015 14:14:46 +0100

Hi Lada,

> I understand the order is fixed in this case.
This makes sense to me, otherwise it would be valid to perversely put 
the 'yang-version "1.1"' statement right at the end of the module 
definition.

The section 14 text states:

    In YANG, almost all statements are unordered.  The ABNF grammar
    [RFC5234] [RFC7405] defines the canonical order.  To improve module
    readability, it is RECOMMENDED that clauses be entered in this order.

    Within the ABNF grammar, unordered statements are marked with
    comments.

So this does mean that the module sections are ordered.  Potentially a 
more explicit statement somewhere in section 7.1. might be helpful.

Thanks,
Rob

---

From: Martin Bjorklund <[email-masked]>
Date: Thu, 22 Oct 2015 15:20:10 +0200 (CEST)

Yes.

> Here it IMO makes little sense to require fixed order.

Correct.  I have added the comment to this statement as well.

/martin