Subject: url clarification

From: Martin Bjorklund <[email-masked]>
Date: Wed, 15 Feb 2006 13:34:26 +0100 (CET)

Hi,

I once asked what it means to lock a remote url.  Now I might have
found the answer.  The text in section 8.8.5 and the schema differ, and
that usually means that the text is correct.  The schema allows a
<url> element as a  <lock> target, but the text in 8.8.5 does not.
Does this mean that you can't lock a url?

(however, the text in 8.8.1 is a bit more generic than 8.8.5)

/martin

--
to unsubscribe send a message to [email-masked] with
the word 'unsubscribe' in a single line as the message text body.
archive: <http://ops.ietf.org/lists/netconf/>

---

From: Phil Shafer <[email-masked]>
Date: Wed, 15 Feb 2006 08:34:53 -0500

>Does this mean that you can't lock a url?

The original idea was that the world was divided into two halves,
databases and files.  Files could be overwritten and fetched, but
did not have/require the additional overhead of being searchable,
replacable, lockable, etc.  So you could copy-config a file, but
not edit-config or lock-config it.  The text and schema should be
corrected to make this clear.

Thanks,
 Phil

---

From: Martin Bjorklund <[email-masked]>
Date: Wed, 15 Feb 2006 14:33:37 +0100 (CET)

Phil Shafer <[email-masked]> wrote:
> replacable, lockable, etc.

This makes sense.

> So you could copy-config a file, but
> not edit-config or lock-config it.

Aha, not even edit-config?  That's explicitly allowed by 8.8.5.1.   

/martin

---

From: Phil Shafer <[email-masked]>
Date: Wed, 15 Feb 2006 09:13:05 -0500

>Aha, not even edit-config?  That's explicitly allowed by 8.8.5.1.   

Hmmm.... that's a bug in the draft IMHO.

Thanks,
 Phil

---

From: Martin Bjorklund <[email-masked]>
Date: Wed, 15 Feb 2006 22:35:57 +0100 (CET)

> Hmmm.... that's a bug in the draft IMHO.

Sorry, I got it wrong.  8.8.5.1 allows <url> instead of <config> in an
<edit-config>. <url> is not allowed as an <edit-config> target.  An
example is given in D.1.5.  (the schema is wrong though).

However, the example in D.1.5 should probably use <copy-config>
instead of <edit-config>, since an <edit-config> w/o operation
attributes (which is used in the example) will use the default
"merge", which may give different results than <copy-config>.  An
alternative is to set default-operation to "replace" on the
<edit-config>.

/martin

---

From: Andy Bierman <[email-masked]>
Date: Sat, 18 Feb 2006 10:22:30 -0800

> alternative is to set default-operation to "replace" on the

Did we ever resolve all the issues here? (A: no)

Here are the operations that use rpcOperationTargetType:

  operation            text-allows-url     notes
  ------------------------------------------------
  edit-config                N               1
  copy-config                Y               2
  delete-config              Y               3
  lock                       N               4
  unlock                     N               4

Notes:

1) The WG never explicitly allowed or disallowed local or remote files
   as the target of an edit-config.  The intent was as Phil described.
   The rationale is that edit-config involves partial editing, which is
   an attribute of configuration databases, not files.

2) An implementation does not have to support remote to remote copies 
via URLs.

3) This is mandatory to support if :url capability is advertised; 
otherwise there
   is no way to delete local files, after they have been copied to the 
device
   via copy-config.

4) It was never the intent of the WG that remote URLs could be locked 
and unlocked
   But this is not explicitly allowed or disallowed.

Summary Issues:

A) Which is right: The text or the XSD?
   Should the edit-config, lock, and unlock XSD definitions change from
   rpcOperationTargetType to configNameType?  This will explicitly
   disallow URL as <target> in these operations.

B) If an agent advertises the :url capability, what operations
   MUST, SHOULD, or MAY it apply to?  What if an agent only
   supports URLs as the <source> parameter and not the <target> parameter
   on some or all operations?

> /martin

Andy

> -

---

From: Martin Bjorklund <[email-masked]>
Date: Mon, 20 Feb 2006 10:14:02 +0100 (CET)

>   operation            text-allows-url     notes

[...]

>    disallow URL as <target> in these operations.

I think the text makes more sense than the XSD.

>    on some or all operations?

One example is the http/https schemes as target to copy-config.  Which
HTTP operation should the agent use?  PUT?  POST?

Another thing which is not clear is what the file pointed to by an url
looks like.  Suppose you send the agent the following command:

   <copy-config>
     <target>
       <url>ftp://ftp.server.com/my.conf</url>
     </target>
     <source>
       <config>
         <foo xmlns="http://my-foo-ns">
           ...
         </foo>
         <bar xmlns="http"//my-bar-ns">
           ...
         </bar>
       </config>
     </source>
    </copy-config>

What will the 'my.conf' file look like?  Like this:

       <config>
         <foo xmlns="http://my-foo-ns">
           ...
         </foo>
         <bar xmlns="http"//my-bar-ns">
           ...
         </bar>
       </config>

This would be a valid XML document, where 'config' does not belong to
any namespace.  Another alternative is the same contents but w/o the
<config> element, but that wouldn't be valid XML.  There are some
other reasonable alterantives as well.  Maybe this also is
implementation-specific?  (with the usual inter-op drawbacks)

/martin

---

From: Andy Bierman <[email-masked]>
Date: Mon, 20 Feb 2006 07:33:30 -0800

> I think the text makes more sense than the XSD.

I can see it both ways.
One one hand, I sure don't want to make it mandatory
for the :url capability to include <url> as a target of edit-config.
On the other, I don't want to prohibit it without reason either.

Section 8.8.5 is the critical text here.
Taken literally, this section says that <url> as
a target for <edit-config> must be supported,
but the target should be a local file.   Lock and unlock
are not allowed (they are not listed in this section).
This part seems kind of broken to me.

It is data model specific what it means to edit a file
instead of configuration datastore. 

I don't think the WG ever really thought through the
details of this section, especially 8.8.5.1. 

The one thing that is clear is that the text in several sections
is out of sync with the XSD wrt/  the <url> element.

> HTTP operation should the agent use?  PUT?  POST?

good question

>          <bar xmlns="http"//my-bar-ns">

The WG decided awhile back to remove the text vs. XML "format"
parameter from these operations.  There is only XML.  (Even if a
text config file is only conceptually wrapped in a top-level element.)

What my.conf looks like is data model dependent.

> implementation-specific?  (with the usual inter-op drawbacks)

Data-model specific, not implementation-specific.
There is a big difference.

Here is what I mean by top-level wrapper:
(Top-level element name doesn't really matter)

<config>
    <my.conf xmlns="http://example.com/config/1.3">
        ascii config file command
        ascii config file command
        ascii config file command
   </my.conf>
</config>

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Mon, 20 Feb 2006 16:54:10 +0100 (CET)

Andy Bierman <[email-masked]> wrote:
> but the target should be a local file.

It says (8.8.5.1) that <url> must be accepted as an alternative to
<config>, i.e. as the "source" of the edit-config command.  And it
must be a local file (for some reason).  The <target> parameter cannot
be an url.

/martin

---

From: Andy Bierman <[email-masked]>
Date: Mon, 20 Feb 2006 08:05:05 -0800

> must be a local file (for some reason).  The <target> parameter cannot

oops -- you're right.

So the text is right and the XSD is wrong.
The "rpcOperationTargetType" needs to be changed
to "configNameType" for edit-config, lock, and unlock.
End of problem.

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Wed, 22 Feb 2006 14:48:05 +0100 (CET)

Hi,

> to "configNameType" for edit-config, lock, and unlock.

I still think that allowing a url as "source" to <edit-config> is a
problem, or at least quite limited.  If it has to be a local file (as
it is now), this file must be created somehow.  D.1.2 shows an example
where the file is created by doing a <copy-config> to the file.  Then
in D.1.5 the file is used in <edit-config>.  This is a "type-mismatch"
problem.  The <config> parameter of <copy-config> is a "complete
configuration datastore", but the <config> parameter of <edit-config>
is not - it's "all or part" of a configuration.

So maybe copy-config to a file is just a dumb verbatim copy?

Then in D.1.3, the incoming file (the "all or part" of a
configuration), is validated.  Is this supposed to work?  I.e. is
validate really supposed to validate anything that can be sent to
<edit-config>?

/martin

---

From: Phil Shafer <[email-masked]>
Date: Wed, 22 Feb 2006 09:16:51 -0500

>So maybe copy-config to a file is just a dumb verbatim copy?

That was the original intention.

>validate really supposed to validate anything that can be sent to

I think we're probably a little loose on this point, since
validate really needs a complete config, but edit-config
does not.

Thanks,
 Phil

---

From: Balazs Lengyel <[email-masked]>
Date: Wed, 22 Feb 2006 15:36:07 +0100

Hello,
Let me quote a recent letter from Andy. He clearly states that validation of inline 
configuration is allowed.

Andy Bierman wrote:
 > Balazs Lengyel wrote:
 >  > Hello,
 >  > - The validate operation might contain inline config as well as I
 >  > understand the XSD. In the text only datastore name is mentioned.
 >  > Which is correct ?
 >
 > inline also -- this bug was already found
 >

Also the netconf-11 draft was changed to include inline validation.

8.6.4.1.  <validate>
      source:
             Name of the configuration datastore being validated, such as
             <candidate> or the <config> element containing the
             configuration subtree to validate.

We intend to use this capability.

Balazs

-- 
Balazs Lengyel                       Ericsson Hungary Ltd.
TSP System Manager
ECN: 831 7320                        Fax: +36 1 4377792
Tel: +36-1-437-7320     email: [email-masked]

---

From: Balazs Lengyel <[email-masked]>
Date: Wed, 22 Feb 2006 15:42:27 +0100

I believe that copy to a file (copy running -> external-URL) would be a very useful 
function. It could be used to backup a complete configuration to an external location (or 
internal).
The same way (copy external-URL -> running) would be very useful for reloading a previous 
backup.

Balazs

---

From: Martin Bjorklund <[email-masked]>
Date: Wed, 22 Feb 2006 16:00:01 +0100 (CET)

Balazs Lengyel <[email-masked]> wrote:

> I believe that copy to a file (copy running -> external-URL) would
> be a very useful function.

This is supported in the protocol.

/martin

---

From: Andy Bierman <[email-masked]>
Date: Wed, 22 Feb 2006 09:38:57 -0800

> is not - it's "all or part" of a configuration.

I disagree.
This is in there by design to support configuration servers.
For example:

  <edit-config>
    <target><running/></target>
    <default-operation>none</default-operation>
    <test-option>test-then-set</test-option>
    <error-option>rollback-on-error</error-option>
    <url>https://example.com/config/myConfig.xml</url>
  </edit-config>

*Note to Rob!!*
There is a bug in the example at the top of page 92 in prot-11.

It shows <url> being used the way it is in every other parameter,
as a child node instead of a replacement node.

  <edit-config>
    <target><running/></target>
    <config>
        <url>https://example.com/config/myConfig.xml</url>
    </config>
  </edit-config>

> So maybe copy-config to a file is just a dumb verbatim copy?

Yes and No -- it depends on the URL scheme and the data models involved.
I don't think the spec needs to get too detailed here.  There is already
plenty of documentation on URL schemes. 

> validate really supposed to validate anything that can be sent to

Yes -- the <config> element just provides the config source
and the validate (via edit-config test-then-set) happens after
the config input is received.

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Wed, 22 Feb 2006 19:21:10 +0100 (CET)

>     <url>https://example.com/config/myConfig.xml</url>

I agree with you that this would be useful.  But the text in 8.8.5.1
says that the url should specfiy a local file.  Maybe that sentence
just should be removed?  I don't quite see the point of that
restriction.

> Yes and No -- it depends on the URL scheme and the data models involved.

When I wrote "copy-config to a file" I meant "copy-config to a local
file as specified by an url with a 'file' scheme".

> plenty of documentation on URL schemes. 

I'm not saying that the draft should be changed, I'm just looking for
clarification.  However, the way the text is written for copy-config,
this behaviour is not very obvious (from 7.3):

      Create or replace an entire configuration datastore with the
      contents of another complete configuration datastore.

> the config input is received.

The example in D.1.3 uses the <validate> command, not edit-config
test-then-set:

       <validate>
         <source>
           <url>file://incoming.conf</url>
         </source>
       </validate>

Phil's reply was that <validate> needs a complete configuration.  Are
you saying that it doesn't?  The text says  "... the <config> element
containing the configuration subtree to validate."

/martin

---

From: Andy Bierman <[email-masked]>
Date: Wed, 22 Feb 2006 12:30:57 -0800

Martin Bjorklund wrote:
> >....

To be picky, it says should instead of SHOULD.
It really doesn't mean anything normative.

It doesn't say you better hope nobody deletes the local or remote
file before the edit-config is done either.  (Since lock is disallowed,
all you can do is hope ;-)

>       contents of another complete configuration datastore.

This means "copy-config is a replace (not a merge) edit operation".

> containing the configuration subtree to validate."

I don't think the text says that anywhere.
Certainly, the inline config XML does not have to be a complete 
configuration.
I don't think we can really say much (in the prot spec) about the content
derived from the URL dereference process.

Andy

---

From: Andy Bierman <[email-masked]>
Date: Wed, 22 Feb 2006 13:46:29 -0800

Martin Bjorklund wrote:
> Yes I noticed that.  Ok, so the intention is that the url can be
> remote in this case.  This makes perfect sense to me.
>
>   
> I should have mentioned the reference, sorry, it's in 8.6.4.1. (My
> point here was that the term "configuration subtree" can be
> interpreted as a partial config.)
>
>   
> Ok.  But then the validation doesn't give you much more than syntactic
> checks.  If you first validate a partial config and then try to apply
> it with edit-config, the resulting configuration might be invalid.
>   

It depends on the data model!
In my models, there is a conceptual difference between
parameter sets and mere complexTypes (unlike XSD BTW).
A parameter set can be validated for referential integrity, but
not external dependencies. 

This is more than syntax-check but less than full-config validate.

BTW, it doesn't say anywhere in the spec that the agent guarantees
the input that passed a <validate> will always work in a subsequent
<edit-config> or <commit>.  It is a best-effort feature.

> So if you do <validate> on the candidate, it will validate the full
> config. 

Yes

>  And <validate> on inline config will validate a partial
> config (e.g. it must allow incomplete relations). 

Yes

>  Then the question
> is what should <validate> on a local file do?  Should it treat it as a
> full config or partial?
>   

IMO, it doesn't really matter where the input comes from.
The agent can validate what it gets against what it has, the best it can.

To repeat myself -- it doesn't really say anywhere in the spec
that URL means MUST be a complete config file, or MUST be
same or different format than the representation of the built-in
configuration datastores.

The <url> feature is just supposed to be input redirection (in this case).

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Wed, 22 Feb 2006 22:28:09 +0100 (CET)

So if you do <validate> on the candidate, it will validate the full
config.  And <validate> on inline config will validate a partial
config (e.g. it must allow incomplete relations).  Then the question
is what should <validate> on a local file do?  Should it treat it as a
full config or partial?

/martin

---

From: Balazs Lengyel <[email-masked]>
Date: Fri, 24 Feb 2006 11:14:00 +0100

Hello
Is it possible or allowed in Netconf to send in a partial config in a validate operation 
and ask the node to validate this subtree in the context of the current running 
configuration. That would be more then validating just the subtree as we could consider 
broader dependencies.
I would like a validation saying that if I issue these changes will the running config 
still be valid ? It would be something like an edit-config operation wit h a test-option 
set to <test-only>.

Is there a standard way to do this ?
(I could do a copy-config to candidate; then edit-config on candidate; then validate, but 
I am looking for a simpler solution without candidate config.)

Balazs