Subject: problems with 'ignore-error' <error-option>

From: Andy Bierman <[email-masked]>
Date: Thu, 02 Feb 2006 08:50:01 -0800

Hi,

We have heard from one implementer that says they cannot
support the ignore-error <error-option> in the <edit-config> operation.
Now one more.

Here is the total documentation on the ignore-error option:

         ignore-error: Continue to process configuration data on error;
            error is recorded and negative response is generated if any
            errors occur.

1) How many operators want this option?

2) How many vendors are supporting this option?

3) What does this option really mean?

   - ignore malformed XML?
     Most tools won't do that.  This is not only dangerous,
     it isn't very easy to do.

   - ignore missing parameters?  What happens if the <config>
     parameter is missing?

   - ignore extra parameters?

   - use RPC parameters that don't pass the schema validation tests?

   - ignore access control violations?

   - ignore lock access violations?

   - ignore unsupported portions of the config parameter sub-tree?

   - use parameters that fail the referential integrity tests?

   - continue to install child nodes of a parent node with errors?
     (How?)

IMO, this mandatory feature of the NETCONF protocol is poorly defined,
hard or impossible to implement, and has no chance of interoperability.
It needs to be fixed or removed from the protocol.

Andy

--
to unsubscribe send a message to [email-masked] with
the word 'unsubscribe' in a single line as the message text body.
archive: <http://ops.ietf.org/lists/netconf/>

---

From: Steve Berl <[email-masked]>
Date: Thu, 02 Feb 2006 09:55:05 -0800

The original intent of the option was to have behavior similar to Cisco IOS
doing "copy tftp://foo/blah running-config". In this case the configuration
is a list of lines, each represents an atomic configuration change
operation. If one line generates an error we can continue on and execute the
remaining lines, ignoring the error. The idea is that commands that don't
make sense can be ignored, but the ones that do make sense can be applied.

This proves to be a convenient behavior when an existing configuration runs
on a different device than it was originally created for, or when a device
has the software upgraded and some older commands are no longer supported,
or if the line cards in a device have changed so that some interfaces that
were there are now missing.

One of the data models that we are supporting in the Cisco implementation of
netconf is a list of command lines and this same behavior can be maintained
with the ignore-error option.

So, to better define the behavior in the netconf spec we need to be clear
about the errors that we are choosing to ignore. I think that these are the
errors that occur in the processing (not the parsing) of the <config>
element in an <edit-config>.

There's probably a better way to say that.

-steve

---

From: Phil Shafer <[email-masked]>
Date: Thu, 02 Feb 2006 13:40:08 -0500

>element in an <edit-config>.

Clarity may be difficult, given the Twilight Zone of failure cases.
If the client gives you a <grue> element but a <grue> is only valid
(or worse, only parsable) if the GRUE FRU is plugged into your box,
the difference between processing errors and parsing errors becomes
pretty vague.  Recovery may just be best effort.

Thanks,
 Phil

---

From: Andy Bierman <[email-masked]>
Date: Thu, 02 Feb 2006 10:48:58 -0800

> make sense can be ignored, but the ones that do make sense can be applied.

Are you talking about simple XML-ifies CLI?

   <config>
      <cli>blah-blah</cli>
      <cli>blah-blah</cli>
   </config>

This is a trivial case.
Real XML data models are not so easy to implement for this feature.

> were there are now missing.

I understand the intent of the feature.
I don't think it is specified in a manner that will
allow developers to create interoperable implementations.

> element in an <edit-config>.

I think you mean the application of config changes.
If a config change passes all tests then apply it,
otherwise go on to the next config change.

I think that's what you mean.
Just because an operation parses doesn't mean resources like locks
are free, or access control rules are met.

> There's probably a better way to say that.

The problem with this feature is that netconf is not a  CLI.
There is no concept in netconf that the XML payload within
an <edit-config> operation is serialized into an integral number
of "lines".  "When an error occurs, continue to the next line"
does not mean anything in XML.

Support of this feature requires a lot of custom work,
and prevents widely available validation tools from being
used (because they validate an entire sub-tree or nothing).

You can't even say  "on error continue at the next sibling node"
because that isn't always true.  All you can say is "it is an
implementation-specific matter if and where processing
of the <config> sub-tree will continue after an error is
encountered."

IMO, this means "no interoperability whatsoever".

Andy

---

From: Steve Berl <[email-masked]>
Date: Thu, 02 Feb 2006 10:48:42 -0800

Perhaps we can leave the specification of what errors are "ignorable" to be
part of the data model description. It seems to be coupled to the way the
contents of the <config> element are processed.

-steve

---

From: Steve Berl <[email-masked]>
Date: Thu, 02 Feb 2006 11:49:59 -0800

See my response to Phil. I propose that the errors that are "ignorable" be
part of the description of the data model as the behavior seems to be
coupled to the contents of the <config> element.

I think we have the same problem with "stop-on-error" as we do with
"ignore-error". In either case there is a partial success and partial
failure of the request. How can we stop-on-error how do we know which parts
of the request worked and which didn't get executed? It is specific to the
data/processing model that is being used.

The whole idea of the error-option depends on the notion that there are
multiple atomic operations described in the <config> element, and that some
may succeed and some may fail. The order in which these atomic operations
are to be attempted may be specified in the content of the <config> element,
or not. In the <cli> case below, the boundaries and order of the atomic
operations is pretty clear. In some other data models it might not be as
obvious, but they may still be there. In order for there to be any way to
understand the meaning of the stop-on-error and ignore-error options the
boundaries of the atomic operations need to be understood.

See some other comments below.

> Are you talking about simple XML-ifies CLI?

It is a simple data model, yet we do support this in our implementation. It
is very useful considering the existing base of tools and knowledge that
understand the CLI.

> allow developers to create interoperable implementations.

I agree. The behavior is specific to the data/processing model that is being
used.

> I think that's what you mean.

Yes. I think that is a better way to say it.

> does not mean anything in XML.

Well, the netconf protocol is supposed to be agnostic to the data/processing
model, so it should be able to work for a model that consists of a list of
text commands that are executed in order.

> used (because they validate an entire sub-tree or nothing).

This is back to the definition of what errors are "ignorable". If we say
that errors that are detected in the XML validation are not ignorable, but
those detected in the application of the configuration changes are
ignorable, then this is no problem. If a particular data model definition
wants to say that no errors are ignorable, then that is fine also.

> IMO, this means "no interoperability whatsoever".

Unless there are interoperable data models, their wasn't any
interoperability at this level anyway.

---

From: Andy Bierman <[email-masked]>
Date: Thu, 02 Feb 2006 12:40:22 -0800

Steve Berl wrote:

So you want to punt the issue to the data model specification?

I still think some text is needed to make it clear that NETCONF
in no way specifies what it means to "continue after an error".  It will be
up to the data model specification to make it clear how two
implementations of the the <edit-config> command for the
same data model will behave in the same way.

The current text does not make it clear at all that we only mean
ignore-errors in the XML under the <config> element.
The term 'ignore-errors' is misleading because sub-commands with
errors are not executed. The term 'continue-on-error' would have
been better, but it's not worth changing now.

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Thu, 02 Feb 2006 22:33:43 +0100 (CET)

> element in an <edit-config>.

Aha!  This makes a lot of sense.  The feature to reload a
configuration which may contain old obsolete config parameters is very
convenient.  We have been talking about how to implement this within
netconf... The text should be updated though - both for stop-on-error
and ignore-error.

However, I think it would make more sense to have this option on
<copy-config> which is the command to use if you want to reload an old
config e.g. from a url.

I've mentioned this one before.  If the agent supports
rollback-on-error, I think that it should be up to the agent to use
that strategy instead of stop-on-error, which the text says is
default.  In which case would a manager actually prefer stop-on-error
before rollback-on-error?

Further, the text says that a "negative" response is generated.  I
assume that it should be "partial-operation".  This should also be
stated, since although an rpc-error was returned, the target
configuration has actaully changed.  A manager must be able to detect
this case unambiguously.

If a full config contains a single now obsolete parameter, the reply
might contain all good elements as <ok-element>, and the single
obsolete one as <err-element>.  Suppose the request was

   <config>
     <top xmlns="...">
       <servers>
         <server>
           <id>1</id>
           <descr>some text</descr>
           <name>www</name>
         </server>
       </servers>
       <interfaces>
         <interface>
           <name>eth0</name>
           <id>2</id>
         <interface>
       </interfaces>
     </top>
   </config>

Now, suppose the interface/name has been changed to ifName, and that
servers/descr has been removed in the current data model.  The
error-info of the partial-operation could be:

   <error-info>
     <ok-element>id</ok-element>
     <bad-element>descr</bad-element>
     <ok-element>name</ok-element>
     <bad-element>name</bad-element>
     <ok-element>id</ok-element>
   </error-info>

or worse:

   <error-info>
     <bad-element>descr</bad-element>
     <bad-element>name</bad-element>
   </error-info>

Imagine what it would look like if the config was big.

If the order of the elements are known to be in the same order as in
the incoming config, and all always present, it might be possible to
figure out what happened, but it's certainly not easy.

It's probably too late to change this now, but maybe the contents of
the error-info could be defined by the data model, and not the
protocol?

/martin

---

From: Andy Bierman <[email-masked]>
Date: Thu, 02 Feb 2006 15:22:55 -0800

> <copy-config> which is the command to use if you want to reload an old

I don't want to start adding features to operations
that didn't already have them.

I just want to clarify the error-option in edit-config.

> before rollback-on-error?

The manager should use rollback-on-error if that's what they want.
They default is only applied when the parameter is missing.
I don't think it's ever a good idea to rely on defaults.
We already decided not to have conditional default values
based on various capabilities.  I think we should stick with that decision.

> configuration has actaully changed.  A manager must be able to detect

It's hard to say what the error responses should be in any arbitrary case.
The data model must make it clear (good luck!).

I agree with you that a partial operation needs to be identified
if some but not all of the config element content is applied.

> the error-info could be defined by the data model, and not the

Not sure what problem you are describing here.
I will have to read it again later.

Andy

---

From: Steve Berl <[email-masked]>
Date: Thu, 02 Feb 2006 15:56:34 -0800

I think I agree with you completely.

It would be clearer if the choices were "stop-on-error",
"continue-on-error", and "rollback-on-error". I'll leave it up to the editor
and WG chair to decide whether to change it or not.

The text should also make clear that these only apply to errors in applying
the changes described in the <config> element. Errors outside of the
<config> element are real errors in the use of the netconf protocol and
cannot be ignored. The result should be no change to the device
configuration.

-steve

---

From: Phil Shafer <[email-masked]>
Date: Thu, 02 Feb 2006 20:19:48 -0500

>and WG chair to decide whether to change it or not.

Sounds good to me.

Thanks,
 Phil

---

From: Andy Bierman <[email-masked]>
Date: Thu, 02 Feb 2006 17:53:15 -0800

> and WG chair to decide whether to change it or not.

I don't mind.  IMO, "ignore-errors" is really confusing.
It's not in the form of an action to take when an error occurs,
like the other 2 choices.

A NETCONF data model should describe its referential integrity
requirements somehow, so that implementers know which instances
of which objects are conceptually coupled.  It does not need to
describe any error-option specific behavior.

The real issue is not how agent implementers will deal with this,
but rather "will applications get consistent error info for
the same set of conditions from one agent to another?"

> cannot be ignored. The result should be no change to the device

Can we say "application layer errors within the <config> element."?
Does that cover it?

Also:

"It is a data-model specific matter how an agent divides
an <edit-config> operation into sub-operations
for the implementation of the 'stop-on-error' and 'continue-on-error'
error-option values.  If one or more of the sub-operations
completes, but one or more sub-operations does not complete (due
to errors), then the agent MUST return a 'partial-operation' error tag
in the <rpc-reply>."

Andy