Subject: [netmod] yang: multiple pattern

From: Juergen Schoenwaelder <[email-masked]>
Date: Fri, 1 Aug 2008 12:28:24 +0200

Hi,

I have looked up the XSD specifications and they clearly state that
multiple pattern are ORed together:

http://www.w3.org/TR/xmlschema-2/#src-multiple-patterns

So in YANG words,

   pattern A;
   pattern B;

is equivalent to

   pattern A|B;

and this is why the YANG team initially decided to not support
multiple patterns. With this insight, it becomes clear that the
example discussed in the thread "range/length and pattern statements"
is not solvable unless we handle pattern different than XSD (which
means confusion of readers and real complexity of translators that
would have to AND the YANG pattern into XSD pattern).

So my conclusion is to keep what YANG does today (only one pattern)
and to document why we do not follow XSD here (or that XSD does not
provide more value here).

/js

-- 
Juergen Schoenwaelder           Jacobs University Bremen gGmbH
Phone: +49 421 200 3587         Campus Ring 1, 28759 Bremen, Germany
Fax:   +49 421 200 3103         <http://www.jacobs-university.de/>
_______________________________________________
netmod mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/netmod

---

From: Andy Bierman <[email-masked]>
Date: Fri, 01 Aug 2008 04:20:18 -0700

> http://www.w3.org/TR/xmlschema-2/#src-multiple-patterns

thanks for taking my action item :-)
This is the section I remember.

> provide more value here).

I want to remain compatible with XSD.
YANG has quoted string concatenation,
so why not just add an OR expression to the pattern
that is already there?

YANG still supports ANDed patterns:

typedef A {
   type string {
     pattern '...';
   }
}

typedef B {
   type A {
     pattern '...';
   }
}

(BTW, it is good practice to use single quoted strings
because some pattern escape sequences get processed
by the YANG string rules, and break the pattern.
SQ strings never get altered by the complier.

> /js

Andy

---

From: Ladislav Lhotka <[email-masked]>
Date: Sat, 02 Aug 2008 20:43:46 +0200

Hi,

in RELAX NG, multiple patterns are ANDed:

http://www.relaxng.org/xsd-20010907.html#IDAMDYR

This was accepted in order to have the functionality available in XSD,
see
http://lists.oasis-open.org/archives/relax-ng/200105/msg00211.html

I guess the issue for YANG is: Is there a way for ANDing patterns? If
not, I'd suggest to follow the RELAX NG way.

Lada

-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C

---

From: Andy Bierman <[email-masked]>
Date: Sat, 02 Aug 2008 11:58:01 -0700

> not, I'd suggest to follow the RELAX NG way.

Yes there already is a way to AND patterns with multiple
typedefs.  You can also split up pattern strings into
as many lines and fragments as you want.

I do not support any plan to become incompatible with XSD,
without a really compelling use-case.  This is redundant
functionality, and therefore does not qualify.

Andy

---

From: Ladislav Lhotka <[email-masked]>
Date: Sat, 02 Aug 2008 21:07:08 +0200

> typedefs.  You can also split up pattern strings into

Fine, then I agree to leave it as it is now.

> as many lines and fragments as you want.

Do you mean using the standard YANG concatenation of strings
'...' + '...' ?

Lada

-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C

---

From: Ladislav Lhotka <[email-masked]>
Date: Sat, 02 Aug 2008 21:09:17 +0200

> is not solvable unless we handle pattern different than XSD (which

It should then be solvable via the nested typedefs.

Lada

-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C

---

From: Andy Bierman <[email-masked]>
Date: Sat, 02 Aug 2008 12:21:43 -0700

> Fine, then I agree to leave it as it is now.

me too -- it is compatible with both XSD and RNG as-is.

> Do you mean using the standard YANG concatenation of strings

yes -- you can even put comments between the pieces.
IMO, this is good enough.  Multiple errors, descriptions,
and references for one pattern seems excessive.
Does the app-developer really need to know which
arbitrary pattern fragment caused the error?

Andy

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Mon, 18 Aug 2008 15:42:28 +0200

> as many lines and fragments as you want.

I just made the experiment of feeding

foo = 
  xsd:string {
    pattern = "bb"
    pattern = "a.*a"
  }

into trang and I got the following output

  <xs:simpleType name="foo">
    <xs:restriction>
      <xs:simpleType>
        <xs:restriction base="xs:string">
          <xs:pattern value="bb"/>
        </xs:restriction>
      </xs:simpleType>
      <xs:pattern value="a.*a"/>
    </xs:restriction>
  </xs:simpleType>

Obviously, they do translate ANDed pattern by generating additional
anonymous types. If we want to support multiple ANDed pattern in YANG,
we could simply do the same. I am not saying we should - just that
there is a relatively straight-forward translation path.

/js

---

From: Andy Bierman <[email-masked]>
Date: Mon, 18 Aug 2008 07:06:14 -0700

> there is a relatively straight-forward translation path.

But this is not correct according to the rules in XSD.
These patterns are supposed to be ORed together.

Andy

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Mon, 18 Aug 2008 16:28:20 +0200

On Mon, Aug 18, 2008 at 07:06:14AM -0700, Andy Bierman wrote:
>> foo =   xsd:string {

I fail to see why this is not correct. Please explain.

/js

---

From: Andy Bierman <[email-masked]>
Date: Mon, 18 Aug 2008 07:48:02 -0700

> I fail to see why this is not correct. Please explain.

oops -- sorry -- I did not see the nested simpleType.

Andy

---

From: David Partain <[email-masked]>
Date: Tue, 19 Aug 2008 12:28:41 +0200

Hi all,

> there is a relatively straight-forward translation path.

So, if I've got it right, we have two choices:

1. leave things like they are (only one pattern)
2. allow multiple patterns and assume that XSD will deal with it as above.

Is there a compelling reason for #2?

I would fall into the leave-things-as-they-are camp.

Cheers,

David

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Tue, 19 Aug 2008 12:36:01 +0200

> Is there a compelling reason for #2?

The reason in favour of multiple ANDed pattern is expressive
power. See Ladislav Lhotka's email:

<http://www.ietf.org/mail-archive/web/netmod/current/msg00358.html>

It does not make much sense to add multiple ORed pattern.

/js

---

From: David Partain <[email-masked]>
Date: Tue, 19 Aug 2008 14:01:19 +0200

> <http://www.ietf.org/mail-archive/web/netmod/current/msg00358.html>

Yep, that makes sense.

>From this thread, it looks like we can maintain compatibility with XSD whether 
we do this or not.  Given that, what's the cost of allowing multiple 
patterns?  If the cost is negligible and the benefit non-negligible, we 
should do it.

Comments?

Cheers,

David

---

From: Phil Shafer <[email-masked]>
Date: Tue, 19 Aug 2008 10:01:54 -0400

>Is there a compelling reason for #2?

My motivation was to allow multiple error information sets,
so one could emit a message inline with the error.

    type foo {
        pattern "[^-]*-[^-]*" {
            error-message "must be a range in the form 'min'-'max'";
        }
        pattern "[0-9]+-.*" {
            error-message "min must be decimal value";
        }
        pattern ".*-[0-9]+" {
            error-message "max must be a decimal value";
        }
    }

hand-wave, hand-wave....

Thanks,
 Phil

---

From: Andy Bierman <[email-masked]>
Date: Tue, 19 Aug 2008 07:24:21 -0700

>             error-message "max must be a decimal value";

How about fixing section E.4 first?  (See my previous mail.)

IMO, this is over-specification.
We really do not need separate error-messages,
error-app-tags, status, description, and reference clauses
for arbitrary pattern fragments.  (What does it mean to
mix obsolete and deprecated fragments with current ones?)

Assuming E.4 gets fixed, one 'pattern-violation' error-app-tag
per leaf is plenty.

Andy

---

From: Andy Bierman <[email-masked]>
Date: Tue, 19 Aug 2008 07:41:32 -0700

> mix obsolete and deprecated fragments with current ones?)

oops -- just 4 clauses (no status).

I don't really care about this change too much.
It seems redundant, which I heard was bad.
I also think this sort of AND expression is not
going to be needed very often.  It seems like
"accept this OR that" is more likely.

Multiple patterns ORed together works with XSD
but not RelaxNG, so we can't use it.  But at least
this is adding new functionality, not redundancy.

Andy

---

From: Ladislav Lhotka <[email-masked]>
Date: Tue, 19 Aug 2008 19:18:26 +0200

> we do this or not.  Given that, what's the cost of allowing multiple 

Translation to RELAX NG will now be straightforward, translation to XSD
will need to use the extra anonymous type as trang does.

> patterns?  If the cost is negligible and the benefit non-negligible, we 

Doing the AND via multiple typedefs in YANG would be quite clumsy and
non-obvious, so I'd say the benefit is non-negligible.

Lada

-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Tue, 19 Aug 2008 23:17:33 +0200

> this is adding new functionality, not redundancy.

He? Two pattern A and B can always be OR combined into the pattern
(A)|(B). Doing an AND combination of two pattern is much much harder
and can lead to pretty obscure pattern. This is why an AND combination
is actually interesting to have.

/js

---

From: Andy Bierman <[email-masked]>
Date: Tue, 19 Aug 2008 15:00:58 -0700

> is actually interesting to have.

You have to use chained data types to do this now,
but AND expressions are supported.  That use case
made more sense to me as well (e.g., loopback
tested added on top of existing InetAddress test).

But fine, I do not care that much.
IMO cut-and-pasting all the sub-clauses is not as bad
as redesigning the pattern-stmt.

Andy

---

From: Ladislav Lhotka <[email-masked]>
Date: Wed, 20 Aug 2008 20:42:12 +0200

> is actually interesting to have.

+1.

Lada

-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C

---

From: Andy Bierman <[email-masked]>
Date: Wed, 20 Aug 2008 11:57:22 -0700

>> is actually interesting to have.

I think I was the only one objecting, and I already gave in.
IMO, put this one in the 'accepted' column.

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Thu, 21 Aug 2008 09:35:52 +0200 (CEST)

> > is actually interesting to have.

I support this change.  It works like in RelaxNG, and it's trivially
supported in XSD.

/martin

---

From: David Partain <[email-masked]>
Date: Fri, 22 Aug 2008 14:41:31 +0200

Greetings all,

>From the discussion in this thread (which starts at 
http://www.ietf.org/mail-archive/web/netmod/current/msg00605.html),
the chairs believe that we have rough consensus to change the YANG 
specification to allow multiple patterns and assume that XSD mappings will 
have to deal with this by using extra anonymous types (as shown in message 
http://www.ietf.org/mail-archive/web/netmod/current/msg00836.html).

If you disagree that this is an accurate assessment of rough consensus, please 
speak up.

Cheers,

David^2