Subject: [Netconf] configuration datastore version number or 'etag'

From: Juergen Schoenwaelder <[email-masked]>
Date: Mon, 26 Jan 2015 09:44:28 +0100

Hi,

I think we talked about this before - I like to have a simple version
number or an 'etag' such that I can easily refer to a specific version
of a configuration datastore (and I can easily check whether my cached
copy of a datastore is still current). What about having a standard
datastore version number or 'etag'?

/js

-- 
Juergen Schoenwaelder           Jacobs University Bremen gGmbH
Phone: +49 421 200 3587         Campus Ring 1, 28759 Bremen, Germany
Fax:   +49 421 200 3103         <http://www.jacobs-university.de/>

---

From: Martin Bjorklund <[email-masked]>
Date: Mon, 26 Jan 2015 10:03:33 +0100 (CET)

> datastore version number or 'etag'?

I am all for it!  Many existing systems have something like this, but
all proprietary solutions.  This feature is extremely useful, and we
have special code for several proprietary mechanisms; a standard
solution would definitely help.

Someone just have to write up a solution proposal ;-)

A simple, straightforward solution, is to augment netconf-monitoring:

  augment /ncm:netconf-state/ncm:datastores/ncm:datastore {
    when "ncm:name = 'running'";  // can be debated

    leaf transaction-id {
      type string;
      description
        "An opaque string that uniquely identifies the last
         transaction towards the datastore.

         A client can use this to detect if the datastore is
         modified.";
    }
  }

One drawback with this as the only solution is that it requires an
additional round trip to retrieve it after a write operation.  So an
extension to this idea is to let the sender return a "transaction-id"
attribute in the <rpc-reply>, if the operation modified running
(edit-config, copy-config, commit, or some other).

/martin

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Mon, 26 Jan 2015 11:17:26 +0100

>          A client can use this to detect if the datastore is

I am not sure about the name transaction-id. For me, this should
primarily be an indicator when the datastore last changed. If this
object is an opaque string, then it might hold something you call a
transaction-id. It could also be a simple yang:date-and-time object,
e.g.

	leaf last-modified {
	     type yang:date-and-time;
	     description
		"The time at which the configuration datastore
		 was last modified.";
	}

This is simple to implement and gives me an order - if I have multiple
database snapshots, I can reason about their order.

I do not see why this should be restricted to running. Would it not
also be useful to track what is in #startup? Of course, a
last-modified and a copy from #running to #startup does not make it
easy to establish that both configs are the same. Perhaps we should
have both, a last-modified timestamp plus a tag that stays the same as
long as the datastore contents stays the same.

> change drawback with this as the only solution is that it requires an

I think we should also augment the netconf-config-change notification.

For edit-config, copy-config, commit (did I forget one?), it would
indeed be nice to return the new value of this leaf. Unfortunately,
these rpcs have no outputs to augment. (Perhaps a lesson to learn
here, empty outputs can be useful for augmentation.) I do not think
this should be something the <rpc-reply> should be concerned with.

/js

---

From: Martin Bjorklund <[email-masked]>
Date: Mon, 26 Jan 2015 12:33:50 +0100 (CET)

> database snapshots, I can reason about their order.

Yes - as long as the clock is not modified...

> also be useful to track what is in #startup?

Correct.

> Of course, a

Ok, as long as we don't require the tag to be the same if the contents
are the same.  This would require some kind of checksum over the
complete db, and that can be expensive.

> I think we should also augment the netconf-config-change notification.

Yes.

> here, empty outputs can be useful for augmentation.)

The reply to edit-config should be something like this:

  <rpc-reply ...>
    <ok/>
    <xxx:last-modified>20015-04-01...</xxx:last-modified>
    <xxx:db-tag>d4e01293a0</xxx:db-tag>
  </rpc-reply>

This should be fine.  And this is perfectly legal YANG:

  augment "/nc:edit-config/nc:output {
    leaf last-modified { ... }
    leaf db-tag { ... }
  }

However, the encoding text in 6020 is probably not quite correct:

   If the RPC operation invocation succeeded, and no output parameters
   are returned, the <rpc-reply> contains a single <ok/> element defined
   in [RFC4741].  If output parameters are returned, they are encoded as
   child elements to the <rpc-reply> element defined in [RFC4741], in
   the same order as they are defined within the "output" statement.

This text should say that if no output parameters are defined in the
"output" statement, then <ok/> is sent.  Then augmented nodes are sent
as usual.

An issue for YANG 1.1 maybe?

/martin

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Mon, 26 Jan 2015 13:23:38 +0100

> complete db, and that can be expensive.

My idea was that operations like a copy-config from <running> to
<startup> would not lead to a new tag but a new date reflecting when
<startup> was modified. Ideally, the tag should not change as long as
content does not change.

>     leaf last-modified { ... }

Isn't it a probem that "/nc:edit-config/nc:output" does not really
exist?

> An issue for YANG 1.1 maybe?

Maybe, but my concern was that the output path does not really exist.

/js

---

From: Martin Bjorklund <[email-masked]>
Date: Mon, 26 Jan 2015 13:26:36 +0100 (CET)

> <startup> was modified. Ideally, the tag should not change as long as

This is ok.

> Isn't it a probem that "/nc:edit-config/nc:output" does not really

No, this is already covered in 6020:

   The "rpc" statement defines an rpc node in the schema tree.  Under
   the rpc node, a schema node with the name "input", and a schema node
   with the name "output" are also defined.  The nodes "input" and
   "output" are defined in the module's namespace.

So the "input" and "output" nodes *always* exist.  The reason for this
is to handle this particular use case.

/martin

---

From: Andy Bierman <[email-masked]>
Date: Mon, 26 Jan 2015 07:18:13 -0800

> datastore version number or 'etag'?

We have talked about this before (since Oct. 2013)

The 'config-id' in section 2.1 of NETCONF-EX specifies this capability
http://www.ietf.org/id/draft-bierman-netconf-efficiency-extensions-02.txt

We also implement extensions to the netconf-config-change to return
the new config-id.  It is also sent in the <hello> to enable config caching
as soon as the session starts.

Why the interest now in making NETCONF more efficient?
NETCONF is for big routers that do not care about wasting bandwidth.
CoMI will use CBOR encoding and will be built for efficiency.

> /js

Andy

> _______________________________________________
> Netconf mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/netconf

---

From: Andy Bierman <[email-masked]>
Date: Mon, 26 Jan 2015 07:33:33 -0800

> <startup> was modified. Ideally, the tag should not change as long as

Our code originally did this but I took it out because there are corner-cases
where it causes problems for the config-id to go backwards.

The running config-id is saved across reboots so it always increments to
a previously unused uint64 when the config changes.  The config-id
should not increment if the server reboots with the same config it had
during the previous run.

>> This should be fine.  And this is perfectly legal YANG:

This is actually illegal according to the XSD Schema going back to RFC 4741.
The server sends <ok> or data, but not both.  <ok> is what it
sends if there are no child nodes of /rpc/output defined and
no errors occurred in the operation.

We have been putting attributes like timestamps in the <rpc-reply>
start-tag for years.  RFC 6241 is clear that the server must return
any attributes is gets in the <rpc>. It is not clear about the server
adding attributes in the <rpc-reply>.   Clients seems to ignore
them quite easily.

Andy

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Mon, 26 Jan 2015 16:37:55 +0100

> We have talked about this before (since Oct. 2013)

[...]

> CoMI will use CBOR encoding and will be built for efficiency.

I am trying to address a requirement in LMAP. All I need is a
reasonably efficient way to refer to a certain version of the
configuration.

/js

---

From: Andy Bierman <[email-masked]>
Date: Mon, 26 Jan 2015 07:50:16 -0800

> reasonably efficient way to refer to a certain version of the

I suggest a uint64-based config-id.
Timestamps cause problems because going backwards can cause
false positives for cache hits via If-Modified-Since, If-Unmodified-Since.
Daylight savings time causes the same problems.

If a config-id goes backwards (which can happen in certain normal
operations) then If-Match and If-None-Match will not cause false positives.

Andy