Subject: Re: [netmod] Why constrain keyref?

From: "Sharon Chisholm" <[email-masked]>
Date: Tue, 22 Jul 2008 15:49:47 -0400

Sorry, I just found Andy made the same comment. My email search tool is
so slow I didn't think it found anything.

Sharon 

-----Original Message-----
From: Chisholm, Sharon (CAR:ZZ00) 
Sent: Tuesday, July 22, 2008 3:49 PM
To: [email-masked]
Subject: Why constrain keyref?

Hi

The yang specification (section 8.8) says

"If the leaf with the keyref type represents configuration, the list
   entry it refers to MUST also represent configuration.  Such a leaf
   puts a constraint on a valid configuration.  In a valid
   configuration, all keyref nodes MUST reference existing list
entries."

Why can't I have a relationship with something that isn't configuration?
For example, it might make sense to model physical hardware as
non-configuration if it can't be modified. Does this mean I can't
associate a logical interface with a physical port?

Sharon Chisholm
Nortel
Ottawa, Ontario
Canada
_______________________________________________
netmod mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/netmod

---

From: "Sharon Chisholm" <[email-masked]>
Date: Tue, 22 Jul 2008 16:05:47 -0400

Hi

Having done the politically correct thing and read the original thread,
it isn't clear to me what the agreed solution was. I think the only
restriction we should place on the keyref is that whatever I point to
must exist. The fact that it never changes or may change is not
relevant. In fact, if I point at something that gets deleted or removed
and this causes an error to be reported then this is a good thing in my
books compared to failing silently

If that is too hard, then only do referential integrity checking 
   1) On creation
   2) When explicitly asked to by some validate command.

Restricting me to just point at configuration is only solving part of
the problem and probably creating a CLR.

Sharon

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf
Of Chisholm, Sharon (CAR:ZZ00)
Sent: Tuesday, July 22, 2008 3:50 PM
To: [email-masked]
Subject: Re: [netmod] Why constrain keyref?

Sharon 

-----Original Message-----
From: Chisholm, Sharon (CAR:ZZ00)
Sent: Tuesday, July 22, 2008 3:49 PM
To: [email-masked]
Subject: Why constrain keyref?

Hi

Sharon Chisholm
Nortel
Ottawa, Ontario
Canada

---

From: Martin Bjorklund <[email-masked]>
Date: Tue, 22 Jul 2008 22:26:43 +0200 (CEST)

Hi,

> books compared to failing silently

The problem is what happens if you have a config keyref which
points to some state object, and the state object is removed by the
device.  Now your running config is invalid.

To answer your original question:

> Does this mean I can't

The answer currently is that you cannot do this association with a
keyref.  You would have to use the same type and describe the
relationship in the description clause.

>    2) When explicitly asked to by some validate command.

I don't think that would work.  If e.g. the running config sometimes
can be invalid but the system still works, what does the validity
check give you?

/martin

---

From: "Randy Presuhn" <[email-masked]>
Date: Tue, 22 Jul 2008 13:34:22 -0700

Hi -

> From: "Martin Bjorklund" <[email-masked]>
> To: <[email-masked]>
> Cc: <[email-masked]>
> Sent: Tuesday, July 22, 2008 1:26 PM
...
> can be invalid but the system still works, what does the validity

Is it time for a reincarnation of draft-presuhn-referent-00.txt?

The bottom line is that the most robust design is one in which
the semantics of unsatisfied references are spelled out as
part of the model.

Randy

---

From: Martin Bjorklund <[email-masked]>
Date: Tue, 22 Jul 2008 22:40:54 +0200 (CEST)

> the semantics of unsatisfied references are spelled out as

Can you provide a use-case for something else than failing validation
(which is what we do today)?

/martin

---

From: "Sharon Chisholm" <[email-masked]>
Date: Tue, 22 Jul 2008 16:48:46 -0400

Hi

Resource relationships are a critical part of the data model and I think
we need to step back and understand the problem a bit better. The
solution we seem to have on the table falls short of solving this
problem.

The DSDL work describes three phases of validation that I think might be
useful here. We could easily define a new verb that does referential
integrity checking if people don't think the existing validate should do
this. They key is that this information is in a machine readable format.
Note that I have off-box tools very interested in being able to get this
sort of information in the Schema definition so they can perform their
own pre-validation before sending the configuration down to the box.

I started to classify the type of information that a keyref would point
to so we could look at each in turn, including the one you listed where
I point at dynamic-static information. But I don't think this is any
different then configured information. Does it really matter if the
operator removed the thing I was point at or the system?

Sharon
-----Original Message-----
From: Martin Bjorklund [mailto:[email-masked]] 
Sent: Tuesday, July 22, 2008 4:27 PM
To: Chisholm, Sharon (CAR:ZZ00)
Cc: [email-masked]
Subject: Re: [netmod] Why constrain keyref?

Hi,

"Sharon Chisholm" <[email-masked]> wrote:
> Hi
> 
> Having done the politically correct thing and read the original 
> thread, it isn't clear to me what the agreed solution was. I think the

> only restriction we should place on the keyref is that whatever I 
> point to must exist. The fact that it never changes or may change is 
> not relevant. In fact, if I point at something that gets deleted or 
> removed and this causes an error to be reported then this is a good 
> thing in my books compared to failing silently

The problem is what happens if you have a config keyref which points to
some state object, and the state object is removed by the device.  Now
your running config is invalid.

>    2) When explicitly asked to by some validate command.

I don't think that would work.  If e.g. the running config sometimes can
be invalid but the system still works, what does the validity check give
you?

/martin

---

From: Balazs Lengyel <[email-masked]>
Date: Tue, 22 Jul 2008 23:06:01 +0200

Hello,
As you can have a "must" pointing to a state variable even today a valid configuration might 
become invalid.

E.g. The sum of reserved bandwidths on an interface must be lower then the available bandwidth 
(state variable). Now suddenly the available bandwidth becomes smaller. The configuration 
became invalid.

Also it is not written anywhere (and should not) that the system itself can not change it's 
configuration, e.g. you pull out an interface card, it get's noticed and reflected by the system.

By the way as I read it the unique constraint can also be put on state data :-)
Balazs

-- 
Balazs Lengyel                       Ericsson Hungary Ltd.
TSP System Manager
ECN: 831 7320                        Fax: +36 1 4377792
Tel: +36-1-437-7320     email: [email-masked]
_______________________________________________
netmod mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/netmod

---

From: Martin Bjorklund <[email-masked]>
Date: Tue, 22 Jul 2008 23:07:14 +0200 (CEST)

"Sharon Chisholm" <[email-masked]> wrote:
> Hi
> 
> Resource relationships are a critical part of the data model

[...]

> They key is that this information is in a machine readable format.

I fully agree.

> We could easily define a new verb that does referential
> this. 

I definitely think the existing validate should do this.

> operator removed the thing I was point at or the system?

The operator will not be able to remove something which is referred to
by a keyref in a valid configuration.

Let's go back to your example of an interface which has a relationship
to a physical port.  For some devices, the physical ports might be
non-expandable, so in this case a keyref to non-config data would work
- if the keyref is valid once it will stay valid.  But some other
systems might support adding/removing ports.  So suppose I have an
interface which refers to port 17, and port 17 is removed.  What value
does the keyref give you in this case?

/martin

---

From: Phil Shafer <[email-masked]>
Date: Tue, 22 Jul 2008 17:07:51 -0400

"Sharon Chisholm" writes:
>useful here.

Their third phase was for full configuration validation, which the
running config should always pass.

>own pre-validation before sending the configuration down to the box.

The concern is that a reference to non-config data will not
be dependable and any validation that uses it will then also
not be dependable.  If my model requires that all configured
interfaces appear in physical PICs, then the removal of one
PIC can invalidate my configuration.

If I remove a PIC and reboot, the config will fail validation at
boot time, leaving me with a few bad choices.  If you remove a PIC
and I do a commit, I see an error about which I know nothing ("Hey,
who broke dc-core-01's config?").  Either way, we've introduced a
source of instability and error.

>But I don't think this is any

Compare this to a firewall filter.  If an interface references a
filter that isn't defined, the next commit (or the <edit-config>
itself) will fail.  The conifguration should be self-consistent
and shouldn't become invalid based on the state of the box.

Thanks,
 Phil

---

From: Martin Bjorklund <[email-masked]>
Date: Tue, 22 Jul 2008 23:16:13 +0200 (CEST)

Balazs Lengyel <[email-masked]> wrote:
> Hello,
> As you can have a "must" pointing to a state variable even today a
> valid configuration might become invalid. 

So we should fix this.  A 'must' on config data should not be allowed
to refer to non-config data.

/martin

---

From: "Randy Presuhn" <[email-masked]>
Date: Tue, 22 Jul 2008 16:31:22 -0700

Hi -

> To: <[email-masked]>
> Cc: <[email-masked]>
> Sent: Tuesday, July 22, 2008 1:40 PM
...
> (which is what we do today)?

A simple case comes from VACM.

One can create a vacmSecurityToGroupEntry with at vacmGroupName
referring to a vacmAccessEntry which does not exist.  VACM will
work just fine.

Suppose we put in some kind of "must exist" constraint.  Then we
would have had to describe what would happen to any vacmSecurityToGroupEntrys
whose vacmGroupName referred to a particular vacmAccessEntry if
the vacmAccessEntry was deleted.  This sort of dependency cascades
through the model, so that iIt is ultimately much simpler and more robust to
say what it means if a pointer points to something which does not
exist or is not in service at the moment, than it would be to automagically
delete or take out of service configuration elements which refer to things
which have gone away.

A more complicated use case comes from a mux I worked on long ago.
That system supported N-for-M redundancy.  For a particular group
of cards, it was necessary to specify which cards would be members of
the backup pool for the primaries.  However, it was *not* necessary that those
backups had to be physically present when the system was configured.
Even if one insisted on all the cards being physically present at configuration
time, it's still necessary for the behaviour to be well-defined as cards are
hot-swapped.  For example, removing a failed backup card doesn't
invalidate the configurations of the primary cards for which that
card is a member of the backup pool.  Pulling out a working (or failed)
line card should not cause that card's configuration to be forgotten - when 
the replacement is inserted, the (new) card will need to be put in service
with appropriate parameters, and certainly without any intervention from
a management system.

Randy

---

From: "Randy Presuhn" <[email-masked]>
Date: Tue, 22 Jul 2008 16:52:57 -0700

Hi -

> From: "Phil Shafer" <[email-masked]>
> To: "Sharon Chisholm" <[email-masked]>
> Cc: <[email-masked]>
> Sent: Tuesday, July 22, 2008 2:07 PM
...
> The conifguration should be self-consistent

In theory,.  But whenever someone says "in theory", they probably
mean "not really."  :-)

Sharon's right about the need to look at this in terms of validation
phases.  Though there is a certain utility to the kind of validation
that can be done in terms of self-consistency, that's hardly
sufficient for many real-world cases.  It doesn't matter what bit
rate I've configured an RS-232 port to use if what's in that slot
is actually an ethernet card.  Part of the difficulty here is that
we're using the words "valid" and "invalid" in several different
senses.  Depending on which part of the larger problem one
is interested in, each of these understandings of the term can
be quite reasonable or, from another perspective, problematic.

Randy

---

From: "Sharon Chisholm" <[email-masked]>
Date: Tue, 22 Jul 2008 20:19:15 -0400

Hi

It seems the problem comes down to the fact that errors during
configuration can be reported directly to the person who made them and
prevented from being applied while errors introduced by removing
hardware need to be reported with event Notifications. But if I remove
the card, aren't things just as broken from an operational perspective
except that I now have no way to detect and report it?

Sharon 

-----Original Message-----
From: Phil Shafer [mailto:[email-masked]] 
Sent: Tuesday, July 22, 2008 5:08 PM
To: Chisholm, Sharon (CAR:ZZ00)
Cc: Martin Bjorklund; [email-masked]
Subject: Re: [netmod] Why constrain keyref?

"Sharon Chisholm" writes:
>The DSDL work describes three phases of validation that I think might 
>be useful here.

>Note that I have off-box tools very interested in being able to get 
>this sort of information in the Schema definition so they can perform 
>their own pre-validation before sending the configuration down to the
box.

The concern is that a reference to non-config data will not be
dependable and any validation that uses it will then also not be
dependable.  If my model requires that all configured interfaces appear
in physical PICs, then the removal of one PIC can invalidate my
configuration.

If I remove a PIC and reboot, the config will fail validation at boot
time, leaving me with a few bad choices.  If you remove a PIC and I do a
commit, I see an error about which I know nothing ("Hey, who broke
dc-core-01's config?").  Either way, we've introduced a source of
instability and error.

>operator removed the thing I was point at or the system?

Compare this to a firewall filter.  If an interface references a filter
that isn't defined, the next commit (or the <edit-config>
itself) will fail.  The conifguration should be self-consistent and
shouldn't become invalid based on the state of the box.

Thanks,
 Phil

---

From: Andy Bierman <[email-masked]>
Date: Tue, 22 Jul 2008 18:05:04 -0700

> except that I now have no way to detect and report it?

What about the larger issue of leafref?
That could solve all these problems.

We seriously overload the keyref, which is one of the
reasons I don't like it.  It does not support most of
the use cases either.

Use cases:

   1) foreign key in a list (e.g. ifIndex)
   2) foreign leaf used as a key in a list
      (e.g., protocolDirLocalIndex)
   3) copy-by-value shadow of some key leaf
      (e.g., my-ifIndex)
   4) copy-by-value shadow of some non-key leaf
      (e.g., my-ifDescr)
   5) pointer to <running> data or config in notifications
      (e.g., IF-MIB linkDown objects:
       ifIndex, ifAdminStatus, ifOperStatus)

IMO, keyref should be used for (1) and (3).
There should also be leafref, which is not constrained
like keyref, for use cases (2), (4), and (5).
Keyref will stay the same. Leafref must point at an
existing leaf.

The hot-swap provisioning use-case cannot be supported
with either data type.  Just use the EntityIndexOrZero
approach instead, which should be good enough.

> Sharon 

Andy

---

From: Phil Shafer <[email-masked]>
Date: Tue, 22 Jul 2008 23:06:50 -0400

>reasons I don't like it.  It does not support most of

Overloaded in what sense?  It only does one thing.

>       ifIndex, ifAdminStatus, ifOperStatus)

Can use give a non-SNMP dude some details behind (2) - (5)?  Googling
for "my-inDescr" only yields lots of perl code.

keyref doesn't do copying (what's copy-on-value mean?), so I don't
see how it could do (3).

Thanks,
 Phil

---

From: Phil Shafer <[email-masked]>
Date: Tue, 22 Jul 2008 23:31:36 -0400

"Sharon Chisholm" writes:
>But if I remove

No breakage at all.  If the interface isn't there (or is the
wrong type), the configuration is ignored.  When the PIC is
inserted, the system sees the PIC, sees the config, and does
The Right Thing.

Operators have used this for eons to allow them to pre-configure
PICs.  They can add configuration for a non-existent PIC, knowing
that the PIC will appear during the next maintenance window.

We call this "ephemeral interface" internally, but I don't know the
external term, or what IOS calls them, but IIRC they support the
same mechanism.

Thanks,
 Phil

---

From: Balazs Lengyel <[email-masked]>
Date: Wed, 23 Jul 2008 11:01:47 +0200

Hello,
But if the operator puts in the wrong type of card, the configuration will become invalid. True?
Balazs

> inserted, the system sees the PIC, sees the config, and does
So keyref would not be used to reference such a PIC.

---

From: "Sharon Chisholm" <[email-masked]>
Date: Wed, 23 Jul 2008 10:04:47 -0400

Hi

Pre-configuration is certainly a use case to consider. It's one where it
would be nice to bake something into the solution early on. I know when
we tried to glue it on afterwards in the Entity MIB (internally), we ran
to some issues. I think the key is be able to differentiate between
things that are configured but not present where this should not be
considered an error and real error conditions. If you can differentiate
this, then referential integrity algorithms might be able to pass on
this case but fail when the keyref points at something that doesn't
exist at all, like it should.

Sharon  

-----Original Message-----
From: Balazs Lengyel [mailto:[email-masked]] 
Sent: Wednesday, July 23, 2008 5:02 AM
To: Phil Shafer
Cc: Chisholm, Sharon (CAR:ZZ00); [email-masked]
Subject: Re: [netmod] Why constrain keyref?

Hello,
But if the operator puts in the wrong type of card, the configuration
will become invalid. True?
Balazs

Phil Shafer wrote:
>> the card, aren't things just as broken from an operational 
>> perspective except that I now have no way to detect and report it?
> 
> No breakage at all.  If the interface isn't there (or is the wrong 
> type), the configuration is ignored.  When the PIC is inserted, the 
> system sees the PIC, sees the config, and does The Right Thing.
> 
> Operators have used this for eons to allow them to pre-configure PICs.

> They can add configuration for a non-existent PIC, knowing that the 
> PIC will appear during the next maintenance window.
> 
> external term, or what IOS calls them, but IIRC they support the same 
> mechanism.
> 
> Thanks,
>  Phil

---

From: Phil Shafer <[email-masked]>
Date: Wed, 23 Jul 2008 10:13:09 -0400

>But if the operator puts in the wrong type of card, the configuration will bec

No, it's ignored.  Only if the right interface type is in the
right slot will it become implemented.

Thanks,
 Phil

---

From: Andy Bierman <[email-masked]>
Date: Wed, 23 Jul 2008 07:34:39 -0700

> for "my-inDescr" only yields lots of perl code.

How about a hybrid interfaces table:

container interfaces {
   list interface {
     key name;

     leaf name {
       type string {
         length "1..1023";
       }
     }

     leaf ifIndex {
       type if:InterfaceIndex;
       config false;
     }

     // rest of new interfaces table
   }
}

container ifTable {
   list ifEntry {
     key ifIndex;

     leaf ifIndex {
        type leafref {
          path "/interfaces/interface/ifIndex";
        }
     }

     // rest of old ifEntry
   }
}

In the new interface table, the name (e.g., 'eth0')
is the key, not ifIndex.  This is an agent-assigned column,
which many vendors persist after a reboot.

ifIndex is not config.  It is not a key
in the table it is defined.  But it is config, and a key,
in many other tables.

> keyref doesn't do copying (what's copy-on-value mean?), so I don't

I thought Martin said that if you edited the keyref,
that does not change the real value, just the keyref.
That is copy-by-value, not copy-by-reference.

Andy

---

From: Phil Shafer <[email-masked]>
Date: Wed, 23 Jul 2008 10:31:21 -0400

"Sharon Chisholm" writes:
>I think the key is be able to differentiate between

Please help me understand your requirement here.  Keyref gives a
contraint, where the leaf refers to another leaf that must exist.
The keyref path must match at least one leaf value (elsewhere in
the data hierarchy) to be valid.  If there is no matching value,
the configuration is not valid.

But you are looking for a softer version, where the constraint
isn't enforced, right?  This isn't referential integrity, since
the integrity is not enforced.  Can you explain where these
soft references are useful?  Are they used to generate possible
values in scenarios where the user can take either these values
or an unconstrained one?

I understand (and love) the hard constraint for referential integrity,
where the config cannot contain references that point to something
that doesn't exist.  This feels normal, and enforces the goal that
configurations should be self-consistent, that dangling references
are bad and need repaired.  But I'm not getting the need for soft
references, since they aren't a constraint.  Are they suggestions
or helpers to guide the users' choice?  Or something else?

Thanks,
 Phil

---

From: "Sharon Chisholm" <[email-masked]>
Date: Wed, 23 Jul 2008 10:44:44 -0400

Hi

I am actually interested in hard constraints. What I was trying to do
was introduce a soft one to handle the use case you referred to. I don't
want any restrictions on what I can point at.

Sharon 

-----Original Message-----
From: Phil Shafer [mailto:[email-masked]] 
Sent: Wednesday, July 23, 2008 10:31 AM
To: Chisholm, Sharon (CAR:ZZ00)
Cc: Balazs Lengyel; [email-masked]
Subject: Re: [netmod] Why constrain keyref?

"Sharon Chisholm" writes:
>I think the key is be able to differentiate between things that are 
>configured but not present where this should not be considered an error

>and real error conditions. If you can differentiate this, then 
>referential integrity algorithms might be able to pass on this case but

>fail when the keyref points at something that doesn't exist at all, 
>like it should.

Please help me understand your requirement here.  Keyref gives a
contraint, where the leaf refers to another leaf that must exist.
The keyref path must match at least one leaf value (elsewhere in the
data hierarchy) to be valid.  If there is no matching value, the
configuration is not valid.

But you are looking for a softer version, where the constraint isn't
enforced, right?  This isn't referential integrity, since the integrity
is not enforced.  Can you explain where these soft references are
useful?  Are they used to generate possible values in scenarios where
the user can take either these values or an unconstrained one?

I understand (and love) the hard constraint for referential integrity,
where the config cannot contain references that point to something that
doesn't exist.  This feels normal, and enforces the goal that
configurations should be self-consistent, that dangling references are
bad and need repaired.  But I'm not getting the need for soft
references, since they aren't a constraint.  Are they suggestions or
helpers to guide the users' choice?  Or something else?

Thanks,
 Phil

---

From: Phil Shafer <[email-masked]>
Date: Wed, 23 Jul 2008 11:21:29 -0400

>which many vendors persist after a reboot.

If your config references non-config data, you are asking for
trouble.  Even though we try to make ifIndex values persist, it's
not perfect and certainly not something you want to bet your box
on.  If you need to swap out the box, for example, these values
are lost.

>in the table it is defined.  But it is config, and a key,

This makes for a bad model.  If you want ifIndex values to persist,
you need to enforce this constraint by making them part of your
configuration.  Or decide that this is a bad idea and start using
interface names instead of ifIndex values.  Given that this is
how most devices work, it's a much more dependable, well worn path.

>That is copy-by-value, not copy-by-reference.

I don't think this is true.  Martin?

Thanks,
 Phil

---

From: Andy Bierman <[email-masked]>
Date: Wed, 23 Jul 2008 08:57:06 -0700

> on.  If you need to swap out the box, for example, these values

I meant ifIndex is the agent-assigned column, not 'name'.
ifIndex is used in lots of tables and CLI commands.
It is defined as read-only in the IF-MIB.  This has
been going on for decades.  Maybe it's a bad idea, but
it is current practice.

The RMON2 protocol directory uses really complex identifiers.
The agent assigns a read-only number for shorthand use
as an index in all the read-only monitoring tables that
report data per protocol encapsulation layer.

> how most devices work, it's a much more dependable, well worn path.

I don't see ifIndex going away any time soon.
There are lots of dev-hours invested in it already.
Persistent ifIndex support is difficult, but the work
has been done already to support it.

<sidebar>
Here is a notification example from 4.2.10:

      notification link-failure {
          description "A link failure has been detected";
          leaf if-index {
              type int32 { range "1 .. max"; }
          }
          leaf if-name {
              type keyref {
                  path "/interfaces/interface/name";
              }
          }
      }

1) At a systems-level, what value is there is replicating
    the SNMP notifications?  IMO, this is pointless.
    We should use 'smidump -f yang IF-MIB' and use linkUp
    and linkDown directly.  (Same goes for SMIv2 data types
    in YANG.)

2) Note that if-name is properly modeling the semantics
    of the notification, but if-index is not.  This notification
    does not contain the real definition of if-name.  It
    contains a reference to the real if-name.  These semantics
    fall out for free in the SMIv2 OBJECTS clause, but they
    are easily lost in YANG notifications.

3) 'config false;' statements are missing from if-index and
    if-name in the example.

</sidebar>

Andy

---

From: Phil Shafer <[email-masked]>
Date: Wed, 23 Jul 2008 12:09:57 -0400

>want any restrictions on what I can point at.

If you have hard constraints that refer to non-config data, you
have config that won't validate due to dynamic conditions (missing
FRUs, restarting subsystems, etc).  If you want references to
non-config data, you can't make them hard.

(a) validation
(b) hard constraints
(c) allow non-config data

You can pick only two.  I want (a) and (b), and don't want (c).

Thanks,
 Phil

---

From: Phil Shafer <[email-masked]>
Date: Wed, 23 Jul 2008 12:22:40 -0400

>In theory,.  But whenever someone says "in theory", they probably

I didn't say "in theory" and I didn't mean "not really".  I meant
"in the (real) products that we (really) ship to (real) operators
who (really) deploy them in the (real) internet".  Really.  ;^)

>It doesn't matter what bit
>senses.

In YANG, these would be two distinct augmentations based on the
interface type.  <ethernet:speed>10m</ethernet:speed> would not be
in conflict with <serial:baud>9600</serial:baud>.

Thanks,
 Phil

---

From: Phil Shafer <[email-masked]>
Date: Wed, 23 Jul 2008 13:27:01 -0400

"Randy Presuhn" writes:
>the vacmAccessEntry was deleted.

This is the logic behind languages that don't require you to declare
variables before using them, like Perl.  And, like Perl, users
revolt and want stronger enforcement to avoid scenarios where they
make a typo and their program stops working.

The same is true for config.  By allowing dangling references, you
allow errors to be introduced.  A user says "fooo" instead of "foo"
and there's no error.  Or they delete "foo" and there's no error.
Users want strong constraints to help detect errors early and to
prevent new hires from making mistakes that cost money.

>This sort of dependency cascades

This is a simple matter, programmatically, once you have keyref.
"set anything that referenced 'foo' to now reference 'goo'" is
a simple xslt script.

>backups had to be physically present when the system was configured.

This is exactly the sort of ephemeral configuration we are talking
about.  If your backup pool lists a card, that card should be
configured (assuming the card has its own config), but it need not
be physically present.  When it arrives, the system configures it
as instructed.  But allowing the pool to reference a card that isn't
configured (or allowing invalid card names) creates a source of
detectable errors.  Which creates a bug report ;^)

>Pulling out a working (or failed)

Amen.  FRUs aren't config.  Config tells the system how it
should behave, and these rules should be self-consistent.

Thanks,
 Phil

---

From: Phil Shafer <[email-masked]>
Date: Wed, 23 Jul 2008 14:02:26 -0400

>ifIndex is used in lots of tables and CLI commands.

I'm no IOS guru, but "lots of .. CLI commands" doesn't jibe with
what I know.  The ifIndex is displayed in output, but it's not
config for JUNOS or IOS and IOS have knobs to force persistence,
but it doesn't give a value, just "persist".

>been going on for decades.  Maybe it's a bad idea, but

Are we really somehow required to carry ifIndex into the NETCONF
world?  Is using interface name not acceptable?

>report data per protocol encapsulation layer.

This is an SNMP hook that need not be carried forward.

>There are lots of dev-hours invested in it already.

Ditto for SNMP, but there's no need to burden NETCONF with SNMP
design choices.  ifIndex make OIDs simpler.  XML doesn't have that
need.  <interface>so-0/0/0.0</interface> is simpler, more direct,
more informative, and is the way config works in the real world.

Thanks,
 Phil

---

From: Andy Bierman <[email-masked]>
Date: Wed, 23 Jul 2008 11:23:34 -0700

> more informative, and is the way config works in the real world.

I agree we should index interfaces by name, but
I also think it would be very useful to create an integrated
multi-protocol NM standards suite, instead of the ad-hoc
free-for-all the IETF is so good at.  For example,
many SMIv2-to-YANG translated lists will expect the
ifIndex and other essential ifTable fields to be available.

Integrating SNMP and YANG management data is a useful
way to get some standard YANG content.  At the current rate of
delivery, the IETF will take about 140 years to have the
number of standard YANG objects to match the number of standard
MIB objects.

Andy

---

From: "Randy Presuhn" <[email-masked]>
Date: Wed, 23 Jul 2008 12:17:16 -0700

Hi -

> To: "Randy Presuhn" <[email-masked]>
> Cc: <[email-masked]>
> Sent: Wednesday, July 23, 2008 10:27 AM
...
> should behave, and these rules should be self-consistent.

But this is the crux of the need to look at validation in terms of
phases.  The configuration for that card is operationally valid IFF
the card inserted into that slot is of a type compatible with that
configuration.  If someone replaces a failed RS-232 mux card with an ethernet
card, the old configuration (which is what the system should have
remembered) won't work.  However, if "referential integrity" causes
that portion of the configuration to be forgotten / deleted, then when
the tech finally does insert the correct card type the system will
still not be fully operational.

The point where we agree is that there are cases where *self*-consistency
constraints are possible and useful.  Where I think we disagree is the
question of just how much ground is covered by such constraints.  If
onw comes from the world of dynamically reconfigurable systems with
a variety of hot-swappable hardware and software components, or
distributed processing elements collaborating through some kind of
network (even if that network is the box's backplane) the "self-
consistency" constraints are much "softer", as someone else put it.
The simply can't be fully evaluated without looking at a hardware
configuration, and possibly with collaboration from other systems.
Detection of such a constraint violation probably will initiate some
hardware / software action, such as telling a tech pull the offending 
card or announcing a run-time configuration conflict, but it
almost never would be correct for it to initiate side-effects like
deleting the "offending" portions of the configuration or declaring it
invalid.  In a sense the configuration is operationally invalid, because
it isn't working at the moment, but it doesn't mean that there is
anything at all wrong with the configuration itself.

Randy

---

From: Phil Shafer <[email-masked]>
Date: Wed, 23 Jul 2008 17:41:35 -0400

"Randy Presuhn" writes:
>phases.

The canmod-dsdl draft defines its phases as:
- fragment phase (is the xml in the operation valid?)
- standard phase (is the entire config valid with the exception of keyrefs?)
- full validation phase (is the entire config valid including keyrefs?)

I don't see the value in avoiding the keyref tests.  If they aren't
enforced at commit time, you are commiting a configuration that
doesn't pass referential integrity tests.  If you don't enforce
these rules during commit, how can you come back and enforce them
later?  Are you seeing these rules as "lint" (where you allow
mistakes during software development and just plan on fixing them
before the project ships)?

To me, the three different phases in the life of config are:
- when you enter it (could the data ever be valid?)
- validation/commit (is the entire config valid?)
- event (how do I do what I need to do?)

At event may be something like:
- sw install (added new IDP module)
- FRU install (added new PIC)
- protocol event (received BGP connection from 10.1.2.3)
- internal event (cron rotates log files)

In each of these events, the system needs instructions to
know how it should perform.  Should it enable IDP?  Should
it put an IP address on the PIC?  Should it close the BGP
connection?  Should it discard old log files?

>The configuration for that card is operationally valid IFF

"forgotten / deleted"?  Who would do this?

Config shouldn't be discarded, but should only be implemented when
it matches the installed hardware.  You can pre-configure multiple
cards, like:

  <interfaces>
      <interface>
          <name>so-0/0/0</name>
          <description>sonet</description>
          ...
      </interface>
      <interface>
          <name>at-0/0/0</name>
          <description>atm</description>
          ...
      </interface>
      <interface>
          <name>ge-0/0/0</name>
          <description>gig-e</description>
          ...
      </interface>
  </interfaces>

All three co-exist peacefully in the configuration, and when
a PIC is inserted, the proper config is selected and implemented
on the PIC.

>consistency" constraints are much "softer", as someone else put it.

The odd part is that I come from that hot-swap world, but I
have strong constraints.  And customers that want stronger
constraint.  Configuration mistakes are expensive.  Avoiding
them saves money.

>anything at all wrong with the configuration itself.

There must be some confusion.  I'm not suggesting deleting
configuration.  I ignore configuration for FRUs that don't
exist.  If you config a sonet card, an ATM card, and a Gig-E
card all in the same slot, the config is will only drive the
configuration of a PIC of that type.

Thanks,
 Phil

---

From: "Randy Presuhn" <[email-masked]>
Date: Wed, 23 Jul 2008 22:46:42 -0700

Hi -

> Sent: Wednesday, July 23, 2008 2:41 PM

There's more.  Additional phases (which may be beyond the scope of
netconf, but which are nonetheless operationally useful) include
validation with respect to the hardware configuration, installed / 
available software, and "peer" components on other systems.

> before the project ships)?

No, my concern is that these tests are both too little and too much,
depending on what one is trying to do.  They are too much when provisioning
a system, and too little when one is concerned whether the proposed
configuration will actually work given the constellation of installed
hardware / software / firmware.

> - validation/commit (is the entire config valid?)

We need to nail down what "valid" means here...

> "forgotten / deleted"?  Who would do this?

No one in their right mind, I hope.
 
> a PIC is inserted, the proper config is selected and implemented

This is an interesting feature. How does the software responsible
for validating the configuration know that this sort of conditional
behaviour is what is intended, and that it's not a configuration error?
I would have thought that this was precisely the kind of thing that
should be rejected by a self-consistency check.

> exist.

*Some* validation of that configuration is still possible, to
the extent that one can figure out what schema to use to validate
it.  But if there's another chunk of configuration that refers
to one of these "contingent" bits that's being "ignored" because
the card isn't there at the moment, what happens?  I still think
it's simplest to just say what it means to have an "unsatisfied"
pointer.

>  If you config a sonet card, an ATM card, and a Gig-E

Concrete example, using my understanding of your framework:

I have a performance monitor piece of software. (let's not call it RMON)
It is configured to monitor specific interface characterstics.
The exact bits might be different for sonet, ATM, and GIG-E.
it would be configured to monitor all the possible cards described
in your example slot above.  This would seem to be a perfectly
valid configuration, even though at least 2 of the three pointers
would be in some sense bogus at any point in time.

Though I can see the value of having this "contingent" configuration
capability, I can also see the value of being able to enforce a
discipline of allowing only a single configuration for a particular
slot - particularly when that slot corresponds to particular physical
connectors, etc.  It's not clear to me how one indicates which is the
desired behaviour for a slot.  Would this be done in Yang or by some
other means?

Randy

---

From: Phil Shafer <[email-masked]>
Date: Thu, 24 Jul 2008 02:40:35 -0400

>it's simplest to just say what it means to have an "unsatisfied"

But how does one determine the when an "unsatified" reference is
just a typo?  Yes, it's simpler for software to say "That's not a
problem and the user will figure it out eventually", but it's not
simpler for the user.  We use languages that require us to declare
variables so that we can detect errors as soon as possible.  Operators
are the same.  Telling them "error: filter 'customer-intput' is not
defined at commit time beats leaving them on their own to find that
extra 't'.

>desired behaviour for a slot.  Would this be done in Yang or by some

The key idea is that the configuration needs to be valid regardless
of what cards are plugged in.  Once you buy this, you buy that you
can't have config reference non-config, since it's dynamic.

It doesn't really matter that you buy into the JUNOS/IOS pre-provisioning
scheme/"contingent configuration".  If you want a rule that says
only one interface can be configured for each card slot, that's
fine.  You are still only constaining the configuration ('must
count(../*/slot[name = .]) == 1'?), and this constraint does not
refer to non-config data.

Thanks,
 Phil

---

From: Ladislav Lhotka <[email-masked]>
Date: Thu, 24 Jul 2008 08:53:38 +0200

> make a typo and their program stops working.

I beg to differ. I don't use Perl anymore but in Python it's the same
and I haven't seen any Python programmer revolting. And Google for
example is essentially a Python shop. It's more an issue for language
lawyers, in practical programming this is very convenient.

Lada

-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C

---

From: Ladislav Lhotka <[email-masked]>
Date: Thu, 24 Jul 2008 09:14:39 +0200

> doesn't pass referential integrity tests.  If you don't enforce

This is a problem only for the "running" configuration. The "candidate"
configuration must allow for referential inconsistencies and "startup"
probably too.

Lada

-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C

---

From: Phil Shafer <[email-masked]>
Date: Thu, 24 Jul 2008 03:33:22 -0400

>lawyers, in practical programming this is very convenient.

Python doesn't have "use strict" yet, so folks use lint-style tools
like PyChecker:

http://pychecker.sourceforge.net/

Hmmm... looks like you can now add "import pychecker.checker" to
your code now, in a more 'use strict' manner.

Point is that no one likes getting bitten by minor dumb mistakes
so we teach computers to catch these for us.  Detecting typos as
early as possible reduces the cost of mistakes and can eliminates
some classes of errors completely.

Thanks,
 Phil

---

From: Phil Shafer <[email-masked]>
Date: Thu, 24 Jul 2008 03:37:45 -0400

>configuration must allow for referential inconsistencies and "startup"

Candidate delays validation until commit time, so you can have
inconsistencies that are resolved before commit.  Startup behaves
in a third way, since you can't edit startup.  What you copy to it
is that you get.  If you copy a config with inconsistencies, you
should avoid rebooting.

Thanks,
 Phil

---

From: Ladislav Lhotka <[email-masked]>
Date: Thu, 24 Jul 2008 09:48:43 +0200

> Candidate delays validation until commit time, so you can have

Grammar, datatype validation and semantic constraints can be performed
on candidate as well. This is exactly what the validation phases are
about.

Lada

-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C

---

From: Ladislav Lhotka <[email-masked]>
Date: Thu, 24 Jul 2008 10:00:41 +0200

> Python doesn't have "use strict" yet, so folks use lint-style tools

People use lint for C code, too. Pychecker buys you nothing here - it
will only tell you that you use an uninitialized variable, but Python
compiler will do the same.

> Point is that no one likes getting bitten by minor dumb mistakes

I prefer getting occassionally bitten by minor dumb mistakes that are
almost always caught by the compiler anyway than being forced to write
declarations for all variables.

Lada

-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C

---

From: Andy Bierman <[email-masked]>
Date: Thu, 24 Jul 2008 08:58:33 -0700

> declarations for all variables.

Actually, the python compiler will accept all kinds
of 'wrong code', such as function calls,
which might be well-formed Python.
The interpreter won't validate that piece of code unless and
until it is called.  Yuch.

As a C programmer new to Python, I much prefer the picky C compiler.
Phil's analogy is actually relevant to YANG.  I would much rather
have to tell the compiler what I am doing, rather than have it
silently guess.

I want keyref to be as picky as possible.
I want a different construct,
for the not-picky use-cases like pre-provisioning.

As I stated in an earlier mail, keyref is overloaded.
In this case, the language is not capable of distinguishing
the object instance constraints desired by the DM writer.

Either we decide the unsupported constraints are not important
(not likely), or we alter the language to properly support
the requirements.

What about adding a boolean to the keyref:

   type keyref {
      path "/some/path";
      constrained true;
   }

Constrained = true:
    path must match existing leaf object, which must be a key
    keyref object must match existing instances of key object
    keyref object must match config-stmt property

Constrained = false:
    path must match existing leaf object, which may be any leaf
    instance must be valid, but may not exist at the moment
    config-stmt property does not have to match

In either case, typos will be detected by the compiler.
The default for constrained == true (current semantics).

Is this less confusing than adding another data type, like leafref?

What about other properties of keyref?
  * Is it OK for a 'current' keyref to point at a 'deprecated'
     or 'obsolete' key?
  * What if the key is controlled by a capability, different
    than the keyref?  Does the keyref need to account for
    capability-partitioned data models?

Andy

---

From: Ladislav Lhotka <[email-masked]>
Date: Thu, 24 Jul 2008 19:08:50 +0200

>     config-stmt property does not have to match

I don't understand this - does it mean that "constrained false;" will
never enforce referential integrity of the keyref?
I would rather prefer to implement validation phases, e.g.,

    type keyref {
       path "/some/path";
       validation-phase full;
    }

Here, missing target of this keyref would result in an error only if the
validation phase is "full", which could be when validating the running
configuration. Otherwise just a warning could be issued.

The validation phases would be ordered like syslog priorities so that
any phase also includes all stricter phases.

Lada

-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C

---

From: Andy Bierman <[email-masked]>
Date: Thu, 24 Jul 2008 10:39:04 -0700

> never enforce referential integrity of the keyref?

Which referential integrity tests do you mean?
Unconstrained means the DM writer does not want to check
that the instance (if specified) matches an existing
instance of the key.  It just has to match a properly
specified instance.

If config data for missing hardware is kept around
in the device, the operator needs to know if it is there or not.
Config data that magically appears and disappears as HW is
inserted or removed is a bad idea.

> any phase also includes all stricter phases.

I do not think this offers any value or precision.
I would rather not introduce external references for 'full',
or 'partial', or whatever.

The NETCONF WG could not reach any agreement on the precise
meaning of the <validate> operation.  In particular, in
database terms,  there is no assurance whatsoever that
a config that passes a <validate> will pass a <commit>
at any arbitrary time.  No 2 agent implementations
are required to do the same tests on the same config fragment.

The problem is just too hard.
Unlike the database world, there is actually a whole
lot of 'stuff' that goes on in an NE config besides
maintaining the database data.  Some config interacts
with the network, besides the additional internal
data structures.  You simply cannot run all the same code
for validate as commit in a router.

Andy

---

From: Ladislav Lhotka <[email-masked]>
Date: Thu, 24 Jul 2008 21:23:32 +0200

> Which referential integrity tests do you mean?

I mean that the key leaf that the keyref points to may not be present.
 
> instance of the key.  It just has to match a properly

What is the difference between "existing instance of the key" and
"properly specified instance"?

> I do not think this offers any value or precision.

It offers flexibility. The same configuration data can be handled
differently in different contexts.

> meaning of the <validate> operation.  In particular, in

I actually see this as a big problem of YANG - a module it clearly
specifies various constraints but it is not clear at all what the object
is that is to be validated against them - and most likely it is not just
a single object (full datastore, get-config reply, rpc, notification).

Lada

-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C

---

From: Andy Bierman <[email-masked]>
Date: Thu, 24 Jul 2008 12:55:10 -0700

> "properly specified instance"?

properly specified means that the value could be
a valid key value, according to the syntax for the leaf.

> a single object (full datastore, get-config reply, rpc, notification).

One paragraph up you are arguing for flexibility.
Now flexibility is a big problem.

What about resource constraints?
The validate phase may not allocate all the internal
data structures in advance and hold them.  It probably
allocates none of them, and uses some heuristic instead
(if it bothers to check at all).

Config changes that may cause PDUs for various protocols
to be generated are never executed in validate phase.

YANG is great for defining the config database contents,
but we can never forget that this is running on a network
element, not a database server.

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Thu, 24 Jul 2008 23:35:52 +0200 (CEST)

> I don't think this is true.  Martin?

I wouldn't use these terms; it's much simpler than that.   A keyref is
a normal leaf that has its own value just like all leafs.  If you
change the value of the leaf, nothing but the leaf is modified.

/martin

---

From: Ladislav Lhotka <[email-masked]>
Date: Fri, 25 Jul 2008 11:22:48 +0200

> Now flexibility is a big problem.

One thing is the ability to validate the data differently in different,
but precisely defined, contexts, and another thing is the fuzziness of
the notion of validity.

> element, not a database server.

I fully agree. YANG modules are often presented as contracts between the
agent and manager. It would then be quite natural to view such a
contract as a protocol that says: if the manager sends message X, it
will receive a reply R(X) and it means this and this. This NETCONF PDU
perspective has the advantage that the representation of data is given -
XML. It may not cover everything what people expect from YANG, but it
would be quite precisely defined and much easier to deal with concepts
like mandatory, defaults etc., and problems of database backends would
be left to implementors.

Lada  

-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C

---

From: Andy Bierman <[email-masked]>
Date: Fri, 25 Jul 2008 07:24:01 -0700

> but precisely defined, contexts, and another thing is the fuzziness of

But the validation steps are not precisely defined
in real implementations.  Maybe on paper for a dirt-simple
device, but not a real router.

> like mandatory, defaults etc., and problems of database backends would

I don't think this would be worthwhile to document in a YANG module.
You could fill 500 pages with protocol PDU interactions.
YANG just needs to describe the 'valid' configuration end-state,
not define every little way an NE config database can break
while it is being edited.

Andy

---

From: Phil Shafer <[email-masked]>
Date: Fri, 25 Jul 2008 10:46:48 -0400

Ladislav Lhotka writes:
>will receive a reply R(X) and it means this and this.

A YANG module defines the final hierarchy of config, so the contract
is more like: if the manager builds content X (in any number of protocol
operations), the device will perform the function F(X).  How you
build the data is less important than the final state.

Thanks,
 Phil

---

From: Phil Shafer <[email-masked]>
Date: Fri, 25 Jul 2008 10:39:09 -0400

>device, but not a real router.

Can you explain this comment?  I'm not following you.

Thanks,
 Phil

---

From: Phil Shafer <[email-masked]>
Date: Fri, 25 Jul 2008 10:57:17 -0400

>a single object (full datastore, get-config reply, rpc, notification).

The spec definitely needs to add/refine words to say when various
elements are required for protocol operations (like keys) and some
words about when the various constraints can be checked (type
constraints when the element is parsed, must constraint when the
containing element's close tag is seen, etc).

Thanks,
 Phil

---

From: Andy Bierman <[email-masked]>
Date: Fri, 25 Jul 2008 08:03:43 -0700

> Can you explain this comment?  I'm not following you.

The actual validation steps performed are totally dependent
on the particular data structure, and possibly platform/version
as well.  Just declaring 'full' or 'partial' validation
is never going to be uniformly applied to every data structure.

Some data structures are dependent on the config of
other devices, or just the state of the network, so validation
of such config data may need to be in synch with data on other
devices.  In other words, passing a <validate> at time t(0)
may not mean you will pass a <commit> at time t(1).

Andy

---

From: Phil Shafer <[email-masked]>
Date: Fri, 25 Jul 2008 11:25:19 -0400

>is never going to be uniformly applied to every data structure.

Totally agree.

>may not mean you will pass a <commit> at time t(1).

Totally disagree.  If your config validation depends on external
factors, then your system is incredibly fragile.

Imagine when your Genius Routing Guy does something a 2pm and the
config works.  But when the midnight operator turns up an interface
during a 2am maintenance window, the config fails for reasons
unrelated to the interface.  The operator is lost.  He calls up
Genius Routing Guy, who isn't happy to be pulled away from This
Month's Awesome video game (what?  you thought we was asleep?) and
immediately starts sticking pins in his Vender Voodoo Doll.

Or the person answering the phone at 1800MYCABLE needs to change
some subscriber service contract from "silver" to "gold".  They
enter the data into some web form, which goes into the billing
database, which opens a change ticket, which triggers an NETCONF
operation to change that subscriber information on the right box.
But instead the phone person sees a message about whatever config
worked for Genius Routing Guy at time t(0) but no longer works at
time t(1).

You can't build dependable automation around a system that is not
dependable.  If the config is valid, it works today, tomorrow, next
year.  It works on this box and on one I hot-swap in for it.  It
works when I pull a PIC, it works when I stuff in a new one.  The
config has to be dependable.  It's the bedrock upon which all this
automation is built.

If we view config validation as shifting sand, we're in for trouble.

Thanks,
 Phil

---

From: Andy Bierman <[email-masked]>
Date: Fri, 25 Jul 2008 08:42:55 -0700

> If we view config validation as shifting sand, we're in for trouble.

I think you misunderstood my comment.
If the router needs 'fresh' data from the network
in order to validate the config, where fresh < (t(1) - t(0)),
then the validation done at commit time will fail if
the data is unavailable at t(1), but it was available
to the <validate> operation at t(0).

Andy

---

From: Phil Shafer <[email-masked]>
Date: Fri, 25 Jul 2008 11:56:44 -0400

>I think you misunderstood my comment.

I don't think I did.  You said that a config that commits successfully
at one point in time may not commit at some later time, due to conditions
outside the configuration.

>to the <validate> operation at t(0).

This will lead to scenarios like I listed.  If you need "'fresh'
data from the network" and can't get it, the validation will work
differently than it will on the next commit.  This means I would
have no expectation that the next commit would work, even if the
previous one did.

When the network changes, my config may no longer commit.  If the
network is unreachable, I can't commit the fix.  If there's a route
flapping, my phone operator can't sell new services.

A device can't depend on anything external during the commit process,
because that external thing won't be there when you need it.

Thanks,
 Phil

---

From: "Randy Presuhn" <[email-masked]>
Date: Fri, 25 Jul 2008 09:03:06 -0700

Hi -

> From: "Andy Bierman" <[email-masked]>
> To: "Phil Shafer" <[email-masked]>
> Cc: <[email-masked]>
> Sent: Friday, July 25, 2008 8:42 AM
...
...

Expanding just a bit...

The "configuration" that needs to be *operationally* validated is more
than the little bit of XML shovelled around by Netconf protocol.  A
network's configuration also includes the specific bits of hardware,
their hardware /  firmware / software versions, what's plugged in
where, which cables go where, what SLAs are in place with the
carriers, and so on.  If one has this complete picture, then the
"shifting sand" metaphor doesn't apply.  But to the extent that all
this configuration data is ignored, ":netconf" is really a misnomer,
and is merely "devconf".  And if all we have validated is the XML
that goes into a device, and not verified that the device has the correct
hardware configuration and cabling and correctly configured peers,
then "shifting sand" is indeed the correct image.

Randy

---

From: Andy Bierman <[email-masked]>
Date: Fri, 25 Jul 2008 09:11:40 -0700

> because that external thing won't be there when you need it.

Here's a simple example:

At time t(0), I set the VLAN assignment for an unused port,
in the candidate config.  It validates just fine.

Later on, after that VLAN was removed from the network by
somebody else, I <commit> the candidate, and it fails.

Andy

---

From: Phil Shafer <[email-masked]>
Date: Fri, 25 Jul 2008 16:45:28 -0400

>somebody else, I <commit> the candidate, and it fails.

So your view is that I shouldn't be able to add a VLAN
to an in-use port if the other side doesn't have that
VLAN configured?  If my device is a CPE router attached
to a customer-owned switch, and they change the config on
their switch, my device config will no longer commit?

Thanks,
 Phil

---

From: Phil Shafer <[email-masked]>
Date: Fri, 25 Jul 2008 16:53:47 -0400

"Randy Presuhn" writes:
>carriers, and so on.

Absolutely.  Device config is only part of network config, and the
network won't work if the whole thing isn't set up correctly.  But
the bit we're discussing is what happens when you make a device's
config depend on transient things like FRU state and the state of
other devices.

If you let the device config fail commit in such scenarios, you
increase the fragility of the device and remove the idea that
incremental changes will succeed incrementally.  If I add a piece
of interface config, I'm not expecting unrelated BGP config to fail.
I can't even rollback my changes and commit.  I'm totally broken.

>If one has this complete picture, then the

This topic is unrelated to XML.  The same issues are present if you
change config via expect-over-cli.  If you can't depend on the same
config to commit twice in a row, you can't build dependable automation.

Thanks,
 Phil

---

From: "Randy Presuhn" <[email-masked]>
Date: Fri, 25 Jul 2008 14:35:40 -0700

Hi -

> Sent: Friday, July 25, 2008 1:53 PM
...
> config to commit twice in a row, you can't build dependable automation.

I agree that the topic is independent of XML.  It arises for *any*
kind of configuration data modeling effort.  But it's not safe to
ignore it just because it's a generic problem.

This discussion is spiraling in on these questions:
  1) what are the expectations when a configuration is proclaimed "valid"
  2) is a deviced allowed to reject the commit of a configuration which
     is valid according to the schema, but which is "operationally" invalid
  3) are the conditions used to determine whether a commit will succeed
     allowed to change over time?

Whatever (1) is, it needs to be clearly stated, particularly regarding
whether the notion of "valid" is defined *purely" in terms of the
schema definition, and can be evaluated by looking only at the
schema and the configuration document.

Whether one likes it or not, (2) is going to happen.
To the extent that things like out-of-memory or required-interface-missing-
from-hardware do change over time, we're also stuck with (3).

If hardware and inter/intra-system dependencies were static, then it would
be reasonable to assume that if a config commits once it will commit again.
But if someone has swapped out the boards for wrong rev-level replacements,
it shouldn't surprise anyone if the commit fails.

Randy

---

From: Martin Bjorklund <[email-masked]>
Date: Fri, 25 Jul 2008 23:47:34 +0200 (CEST)

> schema and the configuration document.

Currently in YANG we give necessary conditions for a configuration to
be valid (keyref, unique, must, min/max elements).  We never say that
they are sufficient.

> from-hardware do change over time, we're also stuck with (3).

Agreed.

/martin