Subject: Re: [netmod] prefixes in groupings

From: Martin Bjorklund <[email-masked]>
Date: Wed, 01 Jul 2009 09:02:24 +0200 (CEST)

Ladislav Lhotka <[email-masked]> wrote:
> Example:
> 
> module xxx {
>     prefix x;
>     namespace "http://example.com/xxx";
>     grouping foo {
>         leaf a {
>             type uint8;
>         }
>     }
> }
> 
> module yyy {
>     prefix y;
>     namespace "http://example.com/yyy";
>     import xxx { prefix x; }
>     grouping bar {
>         container c {
>             must "x:a < y:b";

This is not correct - the uses statement below adds the nodes from the
grouping foo to 'yyy', so it should be:

             must "y:a < y:b";

>             uses x:foo;
>             leaf b {
>                 type uint8;
>             }
>         }
>     }
> }
> 
> module zzz {
>     prefix z;
>     namespace "http://example.com/zzz";
>     import yyy { prefix y; }
>     uses bar;
> }
> 
> So, if I interpret your statement correctly, the following instance is
> valid according to module zzz:
> 
> <c xmlns="http://example.com/zzz">
>   <a>1</a>
>   <b>2</b>
> </c>
> 
> I think such juggling with namespaces is hardly acceptable. Also, it
> contradicts what Andy said earlier:
> "All prefixes need to be resolved in the module where they are appear."

Can you describe how you would like it to work?

/martin

---

From: Ladislav Lhotka <[email-masked]>
Date: Wed, 01 Jul 2009 09:45:52 +0200

> grouping foo to 'yyy', so it should be:

Right, this was my mistake, but still, the prefix "y" is explicit, so it
should be resolved here to NS URI "http://example.com/yyy" and never
change later.

> >     namespace "http://example.com/zzz";

Another typo, this should have been "uses y:bar;".

> Can you describe how you would like it to work?

1. All names with explicit NS prefixes are resolved to QNames in the 
   module where they appear, independently of whether they are in a 
   grouping or not.

2. Inside groupings, unprefixed names are left "namespace-less" and 
   their namespace is set later when the grouping is used in a schema 
   tree.

3. If a grouping is used inside another grouping, the unprefixed names 
   continue to be namespace-less.

So, module yyy from my example should look like this:

module yyy {
    prefix y;
            must "a < b";
            uses x:foo;
            leaf b {
                type uint8;
            }
        }
    }
}

Here, both "a" and "b" still have no namespace.

module zzz {
    prefix z;
    uses y:bar;
}

Here, y:bar is used (instantiated) in the schema tree of module zzz,
which becomes equivalent to:

module zzz {
    prefix z;
        must "z:a < z:b";
        leaf a {
           type uint8;
        }
        leaf b {
           type uint8;
        }
    }
}

Now, "a" and "b" have the NS URI "http://example.com/zzz" and

is valid wrt zzz.

If the must statement in module yyy was

must "y:a < y:b";

it could be satisfied only if grouping bar is used in module yyy, so the
above document wouldn't be valid.

Lada

-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C

---

From: WashamFan <[email-masked]>
Date: Wed, 01 Jul 2009 17:38:34 +0800

Hi,
 
>     module where they appear, independently of whether they are in a 

I would agree with what Martin said before. If the name refers to a node
in the grouping regardless of the NS prefix, it should always refer to the node
in the grouping.

washam

---

From: Ladislav Lhotka <[email-masked]>
Date: Wed, 01 Jul 2009 12:00:11 +0200

> in the grouping regardless of the NS prefix, it should always refer to the node

It may not be always easy to determine which nodes are in the grouping,
because groupings can be nested. Also, in a grouping like this

...
prefix x;

grouping foo {
   container c {
       must "x:a < ../x:above";
       leaf a { ... }
   }
}

it would be pretty confusing if the resolution of the same prefix "x"
was fixed for 'x:above' and postponed for 'x:a'.

Lada

-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C

---

From: WashamFan <[email-masked]>
Date: Wed, 01 Jul 2009 18:10:54 +0800

Hi,

----- Original Message -----
From: Ladislav Lhotka <[email-masked]>
Date: Wednesday, July 1, 2009 6:00 pm
Subject: Re: [netmod] prefixes in groupings
To: WashamFan <[email-masked]>
Cc: Martin Bjorklund <[email-masked]>, [email-masked]

>  was fixed for 'x:above' and postponed for 'x:a'.

I am not sure I follow. In the example, x:a refers to a node in the grouping
whereas x:about refers to some one outside of the grouping. Why confusing?

washam

---

From: Ladislav Lhotka <[email-masked]>
Date: Wed, 01 Jul 2009 12:59:56 +0200

> whereas x:about refers to some one outside of the grouping. Why confusing?

Because the resolution of the same prefix in the same expression depends
on non-trivial context, which is also complicated by the possibility of
augmenting the grouping when it is used.

Lada
 
-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C

---

From: Andy Bierman <[email-masked]>
Date: Wed, 01 Jul 2009 07:49:12 -0700

>    continue to be namespace-less.

This is worse than what we have now.
Although it is possible to construct corner-cases
that are confusing to the naked eye, they are not
confusing to the software that is processing groupings.

The rule for prefixes is rather simple now,
and it should stay that way.

It is critical that groupings remain transparent to
the protocol.  It would be a disaster to have
a must-stmt or a default for an identityref work
one way in plain YANG, but another way if the
DM writer cut-and-paste those objects into a
grouping, and replaced them with a uses-stmt
for that grouping.

E.g.,

   leaf X { type string; }

must always be equivalent to

   grouping A {
      leaf X { type string; }
   }

   uses A;

for any grouping with any contents.

> above document wouldn't be valid.

Andy

---

From: Ladislav Lhotka <[email-masked]>
Date: Thu, 02 Jul 2009 09:58:58 +0200

> and it should stay that way.

So what is this simple rule?

> for any grouping with any contents.

My rules would not break this.

Lada

-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C

---

From: Martin Bjorklund <[email-masked]>
Date: Thu, 02 Jul 2009 22:36:47 +0200 (CEST)

> was fixed for 'x:above' and postponed for 'x:a'.

The point is that you cannot expect to do a simple text substitution
when you implement 'uses'.  For example, my module might import your
module and some other module:

  import foo-module {
      prefix f;
  }
  import other-module {
      prefix x;
  }

  uses foo;

The prefix "x" in the grouping must not be resolved in this context!
This is no different for must expressions than for references to types
or whatever.  For example:

 grouping foo {
    typedef my-type { type int32; }
    container c {
        must "x:a < ../x:above";
        leaf a { type x:my-type }  // legal
    }
 }

So I agree with Andy that the current rules work fine.

/martin

---

From: Ladislav Lhotka <[email-masked]>
Date: Fri, 03 Jul 2009 10:32:56 +0200

>         leaf a { type x:my-type }  // legal

This is all fine, the problem would arise when this grouping 'foo' is
defined in 'other-module' and used in the local module via 'uses x:foo'.
In the schema tree of the local module, 'a' now has the namespace URI of
the local module, so if the prefix 'x' still corresponds to the NS URI
of 'other-module' the 'x:a' name in the must statement cannot match leaf
'a', although with the rules that you stated earlier 'x:a' would match
'a' in 'other-module' where grouping 'foo' is defined.

Lada

-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C

---

From: "tom.petch" <[email-masked]>
Date: Fri, 24 Jul 2009 17:51:11 +0200

Apologies for taking so long to fail to understand this - I was away at the
time.

But....

Martin in an earlier post in the thread said
"If a must/when expression in a grouping refers to a node in the
grouping - with or without prefix - at conceptual evaluation time they
will always refer to the node within the grouping, i.e. with the
namespace of the module where the grouping is used."

Forget the must/when and XPath:-), I cannot reconcile
"at conceptual evaluation time they
will always refer to the node within the grouping"
which suggests that a leaf in a grouping is in the namespace of the module
where the grouping appears, which is what I also get from
" Identifiers appearing inside the
   grouping are resolved relative to the scope in which the grouping is
   defined, not where it is used." from yang-07.

But
" with the namespace of the module where the grouping is used."
tells me the opposite, that it is the namespace of the module
in which the uses statement appears.

or to quote Andy
"The objects themselves will end up with the namespace
of the final 'uses stmt'."

I am not saying one or the other is preferrable, just that I cannot get
a single meaning from words used in this thread and the I-D.

By contrast, I am comfortable with the interpretation of prefixes; there all the
text tells me the same thing.

Tom Petch

----- Original Message -----
From: "Ladislav Lhotka" <[email-masked]>
To: "Martin Bjorklund" <[email-masked]>
Cc: <[email-masked]>
Sent: Friday, July 03, 2009 10:32 AM
Subject: Re: [netmod] prefixes in groupings

> _______________________________________________
> netmod mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/netmod
>

---

From: Ladislav Lhotka <[email-masked]>
Date: Fri, 24 Jul 2009 21:36:23 +0200

Hi Tom,

this ambiguity is due to the fact that the handling of prefixes in
groupings is inconsistent. I tried to propose a remedy:

http://www.ietf.org/mail-archive/web/netmod/current/msg03052.html

In simple terms:

- unprefixed name in a grouping => null namespace URI, this name will 
  adopt the namespace of the module where the grouping is used
- a prefixed name in a grouping always stays in the namespace the 
  prefix is bound to in the module where the grouping is defined.

Lada

-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C

---

From: Andy Bierman <[email-masked]>
Date: Fri, 24 Jul 2009 12:50:54 -0700

>   prefix is bound to in the module where the grouping is defined.

Except this is fatally flawed because it breaks the
"grouping transparency rule".

For any set of sibling objects, it should not make any
difference whether they are declared inline or via a
uses statement.

Prefixes are handled consistently now (IMO)
because all the QNames appearing in the same YANG [sub]module,
will be processed the same way.

We have to resolve QNames as soon as possible,
not as late as possible, when they are used within
a definition.  Assigning a namespace to the exported
definition is not the same thing as resolving the
dependencies within that definition.

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Fri, 24 Jul 2009 23:20:18 +0200 (CEST)

Hi,

I think there are two issues here.  The first is which namespace
nodes in a grouping belong to.  The answer is that it is the namespace
of the module that has the uses statement.  For example:

  module a {
    namespace "http://example.com/a";
    prefix a;
  
    grouping foo {
      leaf x { type int32; }
    }
  }

  module b {
    namespace "http://example.com/b";
    prefix b;
  
    import a { prefix a; }

    uses a:foo;
  }

  module c {
    namespace "http://example.com/c";
    prefix c;
  
    import a { prefix a; }

    uses a:foo;
  }

The leaf x now exists in two XML namespaces, e.g.

  ...
  <config>
    <x xmlns="http://example.com/b">42</x>
    <x xmlns="http://example.com/c">4711</x>
  </config>

The second issue is how prefixes are resolved within groupings.
In the example above, suppose the foo grouping is defined like this
instead:

    grouping foo {
      leaf x {
        type int32;
      }
      leaf y {
        type int32;
        must "current() < a:x";
      }
    }

The notion "a:x" is used to refer to the node "x" within the
grouping.  When the grouping is used in modules b and c, you cannot
just do a syntactical substitution of the grouping contents.  You must
substitue the "resolved structure", where "a:x" is already resolved.

This is consistent with how type names are resolved in this example:

    grouping foo {
      leaf x {
        type int32;
      }
      leaf y {
        type a:my-type;
        must "current() < a:x";
      }
      typedef my-type {
        type int32;
      }
    }

Again, the type of 'y' must be resolved prior to the uses expansion.

/martin

---

From: Ladislav Lhotka <[email-masked]>
Date: Sat, 25 Jul 2009 15:34:00 +0200

> difference whether they are declared inline or via a

It doesn't, if the module writer uses names without NS prefixes.

> dependencies within that definition.

OK, so consider the following module:

module foo {
  namespace "http://example.net/foo";
  prefix foo;
  grouping ggg {
    ...
    must "foo:bar < 42";
    leaf bar { type uint8; }
    ...
  }
  ...
}

Now, resolving the QName as early as possible means that "foo:bar" is
resolved to "{http://example.net/foo}bar". If module "foo" is imported
and grouping "ggg" is then used in another module, this QName should no
more mean the name of the adjacent leaf because the latter assumes the
namespace of the importing module. Using my rules, this would be a
consequence of using the prefixed name. However, if the must expression
was

    must "bar < 42";

it would be guaranteed that "bar" always refers to the adjacent leaf.

Lada
 
-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C

---

From: Andy Bierman <[email-masked]>
Date: Sat, 25 Jul 2009 07:07:59 -0700

> It doesn't, if the module writer uses names without NS prefixes.

Prefixes are not a problem at all.
Perhaps the terminology needs to be better aligned with XML.
It is not the QName that gets carried forward when a
uses-stmt is processed.  It is the expanded-name
that is remembered, not the QName.

http://www.w3.org/TR/xpath#dt-expanded-name

Andy

---

From: "tom.petch" <[email-masked]>
Date: Sun, 26 Jul 2009 21:00:31 +0200

Martin

Many thanks, that is crystal clear.

But...
what I cannot then understand is the sentence in netmod-yang-07, to whit,
 " Identifiers appearing inside the
   grouping are resolved relative to the scope in which the grouping is
   defined, not where it is used."

"Identifiers .. resolved relative to the scope ... is defined"

I see that as telling me when to add a namespace to an identifier, by
prefix or by default, and telling me that that happens where the grouping
is defined, not where it is used so I understand
     grouping foo {
       leaf x { type int32; }
     }
places 'x' with no prefix as being in
    namespace "http://example.com/a";

How am I misreading that sentence?

Tom Petch

---

From: Ladislav Lhotka <[email-masked]>
Date: Mon, 27 Jul 2009 10:00:59 +0200

> http://www.w3.org/TR/xpath#dt-expanded-name

I fully agree, but somehow we still come to different conclusions.
Back to my example: the QName "foo:bar" is resolved in the context of
module "foo" to "{http://example.net/foo}bar", right? (I am using James
Clark's notation for expanded names here.) Then, if the grouping "ggg"
is used in another module, the must constraint will never match due to
the mismatch in namespaces since leaf "bar" now has the namespace URI of
the module in which the grouping was used. Do we agree on this?

Lada   

-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C

---

From: Martin Bjorklund <[email-masked]>
Date: Mon, 27 Jul 2009 10:55:26 +0200 (CEST)

> How am I misreading that sentence?

Here's an expanded example where an identifier is resloved according to
the rule above:

  module a {

    typedef t { ... }

    grouping foo {
      leaf x { type a:t; }
    }
  }

Now, the identifier "a:t" is resolved in the scope where the grouping
is definied, thus it means the type "t" in module "a".  Note that this
does *not* place 'x' in the XML namespace "http://example.com/a" -
nodes are placed in the XML namespace of where the grouping is used.

/martin

---

From: Martin Bjorklund <[email-masked]>
Date: Mon, 27 Jul 2009 11:44:19 +0200 (CEST)

> the module in which the grouping was used. Do we agree on this?

Yes, you have convinced me that this is the correct way to handle
this.  In the grouping:

    grouping foo {
      leaf a {
        type int32;
        must ". < ../b";  // (*)
      }
      leaf b {
        type int32;
      }
    }
   
leafs a and b are not bound to any namespace, so it makes sense to use
no prefix in the must expression at (*).

(I will bring this up as an open issue at the session today)

/martin

---

From: Andy Bierman <[email-masked]>
Date: Tue, 28 Jul 2009 18:30:58 -0700

> Andy Bierman píše v Pá 24. 07. 2009 v 12:50 -0700:
....
>   namespace "http://example.net/foo";

I finally see the problem.
I checked my code, and it turns out
I don't bind the namespace in the grouping for must/when.
Instead, I bind at the uses-stmt context, doing
what the YANG author wants, not what the YANG spec says.

If I implemented this according to the early
binding prefix rules, it does not matter
whether prefixes are used or not:

   must "foo:bar < 42";
   must "bar < 42";

These are equivalent according to YANG rules,
in this example.  IMO, this is an important
concept to maintain.

But it means that all must/when statements inside
groupings can never refer to a node in the grouping,
when the grouping is used outside the original file.

It gets more complicated when the import-by-revision
is used.  Not only is the prefix to namespace mappping
done in the original file (module foo), but the exact revision
of the module is bound at this time as well.

Even if prefixes were forced to be the same everywhere (yeah, right),
this does not help because the revision-date could be different
each time module 'foo' is imported.

This applies to prefixes in default, type, augment, unique
statements as well.  These can be fully resolved in the grouping,
so they are not a problem.

Treating some QNames with early binding and others with late binding
is very confusing.  I agree there is a problem, but so far, no good
solution exists.

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Wed, 29 Jul 2009 10:54:21 +0200 (CEST)

> is very confusing.  I agree there is a problem, but so far, no good

The idea is that all names with prefixes are resolved and "fixed"
where there are defined (in a grouping or not), and names without
prefixes are fixed to the module where there are used.

(a node outside a grouping is used in the module where it is defined).

/martin

---

From: Ladislav Lhotka <[email-masked]>
Date: Wed, 29 Jul 2009 11:56:52 +0200

> where there are defined (in a grouping or not), and names without

Yes, doing it in the module where the grouping is used is problematic
because the prefix may be different.

> prefixes are fixed to the module where there are used.

In most groupings the names in XPath expressions will be unprefixed and
thus marked for "late binding" of the namespace. Otherwise, prefixed
names are handled identically in groupings and outside them.

(It would be even clearer if the default namespace wasn't used outside
groupings because unprefixed names could then appear only in groupings,
but I am not going to raise this issue again).

Lada

-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C

---

From: Andy Bierman <[email-masked]>
Date: Wed, 29 Jul 2009 04:08:36 -0700

> (a node outside a grouping is used in the module where it is defined).

I do not think this is very intuitive at all.
Most people reading a YANG module will not guess
that identifiers without prefixes that happen to
be inside a grouping are special 'late-binders',
and the identifiers with prefixes are 'early-binders'.

Andy