Subject: nested operation attribute interoperability

From: Andy Bierman <[email-masked]>
Date: Mon, 15 May 2006 17:17:56 -0700

Hi,

I am trying to create an matrix to help me optimize code
related to the processing of the nested operation attribute.
It would be nice if vendors handled this in an interoperable manner.

I try to follow the Postel Principle and not generate any
errors unless it is mandatory.  I use containers to make
it easy to apply a 'delete' operation to an entire table,
as well as easier access control configuration.

At any given point in a <config> subtree, there is a 'parent'
edit operation and a potentially different 'child' edit op.
Obviously the parent edit-op has precedence.  The start state
is allowed to be 'none', but once the operation is set, it can
never go back to 'none'.

This table shows each parent --> child edit-op transition,
and whether it is valid or an error.  My question to the WG
is whether this table is correct or not:

child ->  create   merge   replace   delete
parent
    |
 
   none      V        V        V         V
 
  create     x        V-1      V-1       E-1

  merge      V-3      x        V         V-3    

  replace    V-3      V        x         V-3

  delete     E-1      V-2      V-2       x

Notes:

V:    valid, no errors or warnings

V-1:  valid, but create has precedence.
      Only merge-with-nothing or replace-nothing scenarios are valid.

V-2:  valid, but has no effect because delete has precedence

V-3:  valid, but create or delete rules now in affect for next child

E-1:  'bad-attribute' or 'operation-failed' error (not sure which)
      Requirements for create and delete cannot both be met
      in the same subtree, regardless of the data model

  x:  no edit-op change

Andy

--
to unsubscribe send a message to [email-masked] with
the word 'unsubscribe' in a single line as the message text body.
archive: <http://ops.ietf.org/lists/netconf/>

---

From: Martin Bjorklund <[email-masked]>
Date: Tue, 16 May 2006 08:19:06 +0200 (CEST)

> It would be nice if vendors handled this in an interoperable manner.

Absolutely.

> child ->  create   merge   replace   delete

Our table is a bit different, and I think the reason is b/c you have a
different meaning of the operation 'replace' than us (and others such
as juniper).  This has been discussed before (see the thread "Second
Example in section 7.2 of netconf-proto").  Your 'replace' is more of
a 'merge'.  Personally I think it is even more important to have an
interopable interpretation of the meaning of these operations.

Anyway, this would be our table, differencies to yours are marked with
(*):

   none      V        V        V         V
 
  create     x        V-1      E-1(*)    E-1

  merge      V-3      x        V-4(*)    V-3    

  replace    E-1(*)   V        x         E-1(*)

  delete     E-1      V-2      E-1(*)    x

>       in the same subtree, regardless of the data model

V-4:   valid, but replace rule now in affect for next child

For errors (E-1), we use 'bad-attribute'.  (currently with error-type
'application', which I think might be wrong, it should probably be
'protocol'.  Deciding correct error-type is sometimes tricky...)

/martin

---

From: Andy Bierman <[email-masked]>
Date: Tue, 16 May 2006 02:46:29 -0700

> interopable interpretation of the meaning of these operations.

No -- we decided your interpretation of replace is correct.
I am changing my code for replace to match it.

>>       in the same subtree, regardless of the data model

I think your E-1 is wrong.
If the current subtree is empty, then replace is the
same as create.  My E-1 is only valid for merge or
replace if it is also valid for create.

Replace does not fail if there is nothing to replace.
Only delete fails if there is nothing to delete,
or create fails if the data is already there.

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Tue, 16 May 2006 12:23:45 +0200 (CEST)

> I think your E-1 is wrong.

Do you mean the first one (replace when parent is create)?   The only
reason for allowing this is due to "be liberal in what you accept",
right?  I guess we could also allow delete when parent is create by
the same reasoning.

Allowing create under a replace might make sense though, if we want to
be liberal.

I'm happy to change this code to make it interopable.

/martin

---

From: Andy Bierman <[email-masked]>
Date: Tue, 16 May 2006 08:19:45 -0700

>>>>       in the same subtree, regardless of the data model

Both these error codes are wrong for my E-1.
This is clearly a 'data-exists' error.
There is no way you could get to <b> in the example
below unless <a> existed, which violates the 'create' rules.

edit-config <config> contents
------------------------------

<a operation="create">
   <b operation="replace"/>
</a>

Current Config          Result
------------------------------
empty                   ok
<a/>                    data-exists error
<a><b/></a>             data-exists error

> right?  I guess we could also allow delete when parent is create by

Also because there is no text whatsoever in the document
that declares an error condition if 'replace' is invoked
against an empty portion of the target config.

Andy

---

From: Andy Bierman <[email-masked]>
Date: Tue, 16 May 2006 09:45:50 -0700

>....

I was wrong.  There is some text.
The document is rather sloppy in this area.

Here is the operation attribute definition section [p. 37]:

          merge: The configuration data identified by the element
             containing this attribute is merged with the configuration
             at the corresponding level in the configuration datastore
             identified by the target parameter.  This is the default
             behavior.

          replace: The configuration data identified by the element
             containing this attribute replaces any related configuration
             in the configuration datastore identified by the target
             parameter.  Unlike a <copy-config> operation, which replaces
             the entire target configuration, only the configuration
             actually present in the config parameter is affected.

          create: The configuration data identified by the element
             containing this attribute is added to the configuration if
             and only if the configuration data does not already exist on
             the device.  If the configuration data exists, an <rpc-
             error> element is returned with an <error-tag> value of
             data-exists.

          delete: The configuration data identified by the element
             containing this attribute is deleted in the configuration
             datastore identified by the target parameter.

Note that only the 'create' operation defines an error condition here,
but Appendix A does say delete can cause an error.  It treats
replace and delete the same wrt/ data-missing error:

    Tag:         data-exists
    Error-type:  application
    Severity:    error
    Error-info:  none
    Description: Request could not be completed because the relevant
                 data model content already exists. For example,
                 a 'create' operation was attempted on data which
                 already exists.

    Tag:         data-missing
    Error-type:  application
    Severity:    error
    Error-info:  none
    Description: Request could not be completed because the relevant
                 data model content does not exist.  For example,
                 a 'replace' or 'delete' operation was attempted on
                 data which does not exist.

           *** IMO, the 2nd sentence above is wrong. ***

The text on pg 37 for 'replace' says:
             Unlike a <copy-config> operation, which replaces
             the entire target configuration, only the configuration
             actually present in the config parameter is affected.

The text for copy-config on pg 43 says:

       Create or replace an entire configuration datastore with the
       contents of another complete configuration datastore.  If the
       target datastore exists, it is overwritten.  Otherwise, a new one
       is created, if allowed.

Given the other definitions of 'create' and 'replace', this text
is rather confusing.

---------------------

History:

The WG started out with only merge and replace.
Merge was existing CLI behavior.
Replace was delete-if-it-exists, and then re-create it.
Replace was meant to simplify CLI like:
   no access-list 101
   access-list 101 ....

Then some WG members decided these 2 operations were not
sufficient precisely because they did not generate errors
in these corner-cases.  So we added create (only if it
does not exist) and delete (only if it exists).

I do not remember ever changing the definition of replace
to cause an error like delete.  I remember the opposite.
It is supposed to work like copy-config, except just
on the specified config subset.  IMO, the 2 words 'replace or'
in Appendix A are incorrect.

Andy

---

From: Phil Shafer <[email-masked]>
Date: Tue, 16 May 2006 12:57:51 -0400

>to cause an error like delete.  I remember the opposite.

This is my memory as well, and I concur that the 2nd sentence
is wrong.

Thanks,
 Phil

---

From: Andy Bierman <[email-masked]>
Date: Tue, 16 May 2006 10:44:44 -0700

> This is my memory as well, and I concur that the 2nd sentence

Here is an 'operational sniff test':

Let's say I want to delete all the <user> entries,
but I don't want to remove the <users> container.
Otherwise the purpose of the container is lost.
(OK -- I would probably not really be deleting the user table ;-)

The default-operation parameter for the <edit-config> operation
is set to 'none', although any value actually works in
this example.

1) This causes an error if <users> is already empty

    <users>
       <user operation="delete"/>
    </users>

2) This does not cause an error if <users> is already empty
    and it is simpler to encode:

    <users operation="replace"/>

Both mechanisms should be supported, and they are if 'replace'
is interpreted as originally intended.

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Tue, 16 May 2006 20:36:19 +0200 (CEST)

> edit-config <config> contents

Ah... got it.  We would actually also generate a data-exist error in
this case (if <a> doesn't exist).

Suppose <a/> exists.  Then comes:

<a operation="delete">
   <b operation="merge"/>
</a>

What would the result be?

/martin

---

From: Andy Bierman <[email-masked]>
Date: Tue, 16 May 2006 12:05:19 -0700

> What would the result be?

Hmmm... I think I see your point.
If I apply the delete test to <b> before fully
processing <b>, then I will generate a data-missing
error (if <b> does not yet exist).  But if I process
the nodes fully, bottom up, then <b> always exists when the
delete-test is performed and there is no error.
But this is much harder then processing nodes top-down.
This is also pointless since <a operation="delete"/>
will provide the same functionality.

The result should be no error and <a> is deleted,
but I could also see how a data-missing error
could be generated instead.

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Tue, 16 May 2006 21:31:35 +0200 (CEST)

I agree that top-down processing makes most sense (that's what we do
today). 

So, what should the table be?  Like this maybe

 none      V        V        V         V

create     x        V        V         E-1

merge      V        x        V         V

replace    V        V        x         E-1

delete     E-1      V        E-1       x

The E-1 on delete under replace would be motivated by the fact that
replace is first delete, then create.  So it should give the same
error as delete under create.

The only question mark is maybe the last row.   Why should merge under
delete be allowed?  (maybe for convenience; default operation is
'merge', and with this you can do:

   <user operation="delete">
      <name>fred</name>
   </user>

to delete the fred user, if <name> is used as key.)

/martin

---

From: Andy Bierman <[email-masked]>
Date: Tue, 16 May 2006 14:50:30 -0700

> I agree that top-down processing makes most sense (that's what we do

Let's approach the problem that way.
Let's remember (I didn't) that it is the contents
of the target configuration that matters, not the
child nodes within the PDU request, when testing the
edit operation for the current node.

> child ->  create   merge   replace   delete

merge and replace are not really different in these nesting cases.
Your E-1 for replace-delete and delete-replace seem wrong
because there is no existence test for replace.

>    <user operation="delete">

No -- operations affect the current node and all its children.
The operation for <name> is 'delete'.

Top-down processing works:
If there are no <user> nodes in the target data model, then it
doesn't matter if <name> is present or not -- it is still a
data-missing error as soon as <user> is processed.

Since the operation in effect at the next node is a delete on the
selection node 'fred', if the <user> node did exist, but the entry
for 'fred' did not exist, the delete test for name=fred fails,
and a data-missing error is correctly generated.

If you really did have:

<user operation="delete">
    <name operation="replace">fred</name>
</user>

This 'replace' would be a NO-OP (V-2 as I interpreted it).
Since 'replace' and 'merge' do not have existence test errors,
there is no error generated even if the 'replace' attribute
is processed.  The end result is that the target data model
is deleted, starting from the node that 'delete' is first seen.
(Further processing for selection nodes is required of course.)

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Wed, 17 May 2006 10:55:54 +0200 (CEST)

> because there is no existence test for replace.

If replace-delete is interpreted as "first replace parent. then, while
doing the replace, delete the child", the "delete the child" will fail
with data-missing.

delete-replace... first delete the parent, then replace the child.  If
this does not generate an error, the child would actually be created.
Same goes for delete-merge if this interpretation is used.  So maybe
an error should be generated even for delete-merge.

> The operation for <name> is 'delete'.

You're right.

(But is this obvious from the text?  Is it this sentence from 7.2 that
defines this behaviour:

    The attribute identifies the point in the configuration to perform
    the operation,
)

> (Further processing for selection nodes is required of course.)

But this must also depend on the data model.

If name is key in user, this would delete the user fred:

But this wouldn't delete all admin users, would it? 

    <type>admin</type>
</user>

Unless you apply some kind of subtree filtering semantics on the
delete.

/martin

---

From: Andy Bierman <[email-masked]>
Date: Wed, 17 May 2006 13:51:11 -0700

> an error should be generated even for delete-merge.

1) only 'create' and 'delete' have any kind of existence tests.
    Merge and replace can always work and never generate an error because
    of data present or missing in the target configuration.

2) The 'create' and 'delete' tests apply to the target data models,
    not to the content of the edit-config PDU.  The test is not done
    against the PDU at all.  Doing the tests bottom-up will not change
    anything, because the target data models have not been modified yet.

3) Any operations under a 'delete' within the PDU are academic.
    They only serve to complicate test-suites, and have no operational
    value.  What matters is the state the target config when you are done.
    If an operation to delete <foo> is going to succeed, then
    the 'activity' in the PDU for children of <foo> is irrelevant.
    (Although not to the error reporting rules.)

> But this wouldn't delete all admin users, would it? 

There is nothing in the protocol that actually says the
construct above means "delete the entry named fred".
Your agent just has to know to look for the naming elements
if the NMS is deleting multiple-instanced nodes this way.

> Unless you apply some kind of subtree filtering semantics on the

There are no data modeling rules yet, so you can experiment
and figure out what works best.  I don't see why you couldn't
do something like this.  However, this is not a subtree filter,
and the agent is not obliged to treat it as one.

I was the one who originally pushed hard against the idea
of multiple operation attributes in the <config> subtree, because
I knew there were many more unintended consequences than
benefits in this design approach.

I think you were the one that suggested using xpath expressions
instead of these subtrees. E.g.,

  <rpc>
    <delete target="/users/user[name='fred']"/>
  </rpc>

I am starting to think it would be easier to constrain xpath
than the subtree approach, in order to minimize the dev-effort required
to properly report corner-case errors, and reduce the number
of operationally worthless corner-cases.

Andy

---

From: Andy Bierman <[email-masked]>
Date: Thu, 18 May 2006 18:16:38 -0700

>...

I'm wrong again. :-(
There is another undocumented existence test. (!)
If the default-operation is 'none' for any of the
start nodes in the <config> data, and those corresponding
nodes do not exist in the target, then a data-missing
error should be generated.

[This issue is more relevant for nested dynamic tables,
  but this simple example will suffice]

   <default-operation>none</default-operation>
   <config>
     <users>
       <user operation="create">
          ... rest of entry
       </user>
     </users>
   </config>

If the <users> container does not exist in the agent yet,
then the agent is not supposed to create it, according to this PDU.
The 'none' start state was added as a feature because
the create or delete attributes sometimes must be located
in a child node and not apply to the parent.  This is needed
for access control as well as accurate operation attribute
existence tests.

Now I have to start an errata list :-(

1) data-exists error is for delete only, not replace and delete

2) data-missing error is for create and none, not just create

Note that these errors return no parameters.  I am adding
proprietary elements to identify what is or isn't missing.
Since multiple errors can occur in different sub-trees,
the rpc-error is mostly worthless without them.  This
is not the same as access-denied.  That gets checked before
any other processing of course, and suppresses any more
error messages for the restricted data.

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Fri, 19 May 2006 11:18:46 +0200 (CEST)

> error should be generated.

But this is documented already in section 7.2 in the text about 'none':

         none: The target datastore is unaffected by the configuration
            in the <config> parameter, unless and until the incoming
            configuration data uses the "operation" attribute to request
            a different operation.  If the configuration in the <config>
            parameter contains data for which there is not a
            corresponding level in the target datastore, an <rpc-error>
            is returned with an <error-tag> value of data-missing.

> 2) data-missing error is for create and none, not just create

I think you mean

  1) data-missing error is for delete and none, not delete and replace.

(data-exists is for create only, which is what the draft says).

> proprietary elements to identify what is or isn't missing.

You can always use the standard error-path, even if there's no
error-info, right?  That's what I do, and I also send an error-info
with a bad-element, although it's not required by the description in
the appendix.   (Of course this depends on how you interpret
"Error-info:  none" - does it mean "you MUST NOT send an error-info"
or "you don't have to send an error-info")

/martin