Subject: [netmod] Summary of discussion from netmod interim on i2rs requirements on netmod/netconf

From: Jeffrey Haas <[email-masked]>
Date: Mon, 22 Sep 2014 17:05:46 -0400

[BCC: netconf]

The netmod interim in New York City last week was a productive discussion for
I2RS, although it was frustrating in its conclusions.  (Those conclusions
mostly being, "There's mostly no work needed here, go talk to netconf!")
However, given the large overlap of WG membership between netmod and
netconf, it should hopefully make the next form of this meeting somewhat
easier to have.

Thanks to Cisco for providing the venue for the interim.

This writeup is presented to document my perception of our discussion and
provide the seed to start further discussion in i2rs.  No specific requests
are being made to netconf at this point - and to reduce accidental noise as
we try to converge on our discussion points, I have placed them in the BCC
field.  We can make more formal requests once the i2rs discussion of the
netmod interim has gotten some traction.

Juergen has placed the full netmod minutes here:
http://svn.tools.ietf.org/svn/wg/netmod/yang-1.1/netmod-2014-09-18-minutes.txt

A small excerpt from those minutes:
: Conclusion: We do not need any changes for YANG 1.1 (hurray) but
: instead we need a document that introduces ephemeral datastores and
: clarifies what validation means with ephemeral datastores. In
: addition, the NETCONF WG needs to do work to define suitable
: protocol operations and RESTCONF needs to make sure it is prepared
: to support ephemeral datastores. There is also NETCONF work to be
: done to improve notification handling. The only YANG 1.1 homework is
: to make sure that the language in the YANG 1.1 specification is
: flexible enough to enable the introduction of ephemeral datastores.

Background:

The seed for the discussion at this interim was the somewhat hastily
authored document:
https://tools.ietf.org/html/draft-haas-i2rs-netmod-netconf-requirements-00

-----

Ephemeral state:

The I2RS portion of the discussion was kicked off by talking about the
ephemeral state that I2RS wants to have and why it is important that it is
ephemeral.  The three options from section 2.1 were presented and discussed
at length.  The points about the advantages of a separate datastore (easy
cleanup, clear segregation of state) were compared against the desire of
other non-I2RS users wanting ephemeral state.  A point that was raised early
was, "why can't we simply let the operators choose whether something is
ephemeral or not?"

This lead to a discussion regarding the nature of "priority" and what its
impact was with regard to how ephemeral state was kept.  One thing that
became apparent during this discussion is the I2RS Architecture draft is
potentially not clear enough about the distinction of I2RS state priority
vs. Local Config priority (section 6.3 of the Architecture draft) and client
priority (section 7.8); the word priority is overloaded.

Clarifying client priority vs. state priority lead further into where such
data is kept and how it is used.  Room discussion eventually went to
identity and how it was associated with priority.  An observation was made
that this information is effectively meta-state being kept on the
configuration nodes.  Such information, not being yang specific, was
suggested as a point for netconf.  It was observed that the identity may be
useful information for general users and may not be i2rs specific.  The
analogy was made to "git blame" (and other similar source-control annotation
mechanisms).

The "where is the ephemeral state kept" conversation began to converge
around a possible fourth option - one that apparently had some discussion
much earlier in the i2rs working group's life, apparently.  Here's my
attempt to describe the conclusions of that discussion:

- There is a new ephemeral datastore.
- It may be used by my many applications, including I2RS.
- It has the property that the schema of config state is "visible" in it
  from a read-only perspective.
- It has the property that writes to it may either create new state that is
  disjoint from the config data store state or it may cause changes to
  information that is/was visible from the config state.  Such changes may
  be merge, override, delete, e.g.
- This "shadowing" behavior avoids issues in the existing yang by not
  requiring additional language semantics that would require mechanisms to
  cross-reference between datastores.  (One might observe this is somewhat
  similar to Object Oriented programming where the configuration state
  serves as a "base class" with ephemeral configuration extending that
  base.  A different analogy is the union filesystem semantic.)

Issues that came up with this model under discussion:

- Validation is a problem with this design.  (And was likely even more
  complicated in the original 3 discussion points.  This was one of the
  elements that drove us to this fourth option.)
- It is possible to refer to config state from ephemeral state for
  conditions (must, when), but not vice-versa.
- We (i2rs) must decide what happens when ephemeral state has conditions
  that are invalidated by a change to the underlying config state.  In such
  a case, the config state may validate, but i2rs may not.  Does the
  operation succeed or fail?
- Such constraints and their validation impact may negatively impact the
  desired speed properties of i2rs.  However, it was also observed this is
  only the case when such references exist and that "fast" i2rs operations
  could simply be modeled with no dependencies on config state.

- Ephemeral state priority vs. Local Config priority is difficult to fully
  support in such a model.  In the above discussion, ephemeral state always
  has priority over Local Config.  
- Providing for Local Config to have lower priority may be possible, but
  introduces some unusual semantics.
- Does I2RS have clear use cases for Local Config winning?  The example of
  "configuration of last resort" was offered but is potentially a weak case.

In summary, the fourth option was strongly driven by the arguments of "what
keeps things simple".  The original three options were apparently not simple
enough and there's some doubt about the impacts of the fourth.

Possible yang 1.1 implications:  If the above model works, the discussed
configuration semantic may be the previously discussed
"config (false|true|ephemeral);"

------

Mutual authentication was also discussed.  As noted on list conversation
after draft-haas-* was published, while restconf doesn't currently require
it, it is suspected that it will have such a requirement before working
through IESG.

I suggest that I2RS requirements are addressed for this point.

-----

Identity requirements for the primary identity are covered by authentication
requirements for the existing protocols.  The "secondary identity"
requirement is a netconf issues and we should talk to them about the
implications.  Since the primary requirement for this feature is for
auditing, this may simply be meta-data that is kept on configuration state.
(See commentary above.)

It is noted, however, that introducing something like a secondary identity
would require changes to SSH and/or TLS and may be somewhat difficult to
make the case to the owning working groups.

Per-client priority effectively becomes the ability to say "you can't write
to this if the state exists and has an owner with a higher priority".
Further discussion on this point is needed in light of the fourth option we
were presented.

------

Persistance of subscriptions.  Effectively we were told "go talk to netconf,
it's out of scope for yang 1.1".

Some discussion was given to the filtering considerations.  Extending the
filtering options of the ietf-inet-types module may be appropriate.
[Juergen, is this an action item for yang 1.1?]

-----

Hopefully this is clear enough to kick off discussion.

-- Jeff

---

From: "Bert Wijnen (IETF)" <[email-masked]>
Date: Tue, 23 Sep 2014 09:50:38 +0200

[bcc to i2rs and netmod];

I propose to discuss the netconf aspects of this on the netconf mailing list.
However, for i2rs "consensus" on the specific netconf requirements, I think
it MUSR be dealth with on i2rs list.

Thanks Jeff, this is good for netconf participants to be aware of.

And to discuss/comment on.

Bert

On 22/09/14 23:05, Jeffrey Haas wrote:
> _______________________________________________
> Netconf mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/netconf
>

---

From: "Susan Hares" <[email-masked]>
Date: Tue, 23 Sep 2014 18:14:16 -0400

Jeff:

The fourth option is interesting and creative. A few questions/requests: 1)
How would the fourth model be expressed in the yang model  - config
(empheral)?  2)  Please define your definition of local config with an
example., and 3) please give an example of the (must, when) use cases
envisioned?

Sue Hares 

-----Original Message-----
From: i2rs [mailto:[email-masked]] On Behalf Of Jeffrey Haas
Sent: Monday, September 22, 2014 5:06 PM
To: [email-masked]; [email-masked]
Subject: [i2rs] Summary of discussion from netmod interim on i2rs
requirements on netmod/netconf

[BCC: netconf]

The netmod interim in New York City last week was a productive discussion
for I2RS, although it was frustrating in its conclusions.  (Those
conclusions mostly being, "There's mostly no work needed here, go talk to
netconf!") However, given the large overlap of WG membership between netmod
and netconf, it should hopefully make the next form of this meeting somewhat
easier to have.

Juergen has placed the full netmod minutes here:
http://svn.tools.ietf.org/svn/wg/netmod/yang-1.1/netmod-2014-09-18-minutes.t
xt

Background:

-----

Ephemeral state:

Possible yang 1.1 implications:  If the above model works, the discussed
configuration semantic may be the previously discussed "config
(false|true|ephemeral);"

------

-----

------

-----

-- Jeff

_______________________________________________
i2rs mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/i2rs

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Wed, 24 Sep 2014 09:00:26 +0200

>   base.  A different analogy is the union filesystem semantic.)

I believe the OO analogy is largely mis-leading, I think the union
filesystem semantics are much closer.
 
> "config (false|true|ephemeral);"

My understanding that both the config datastore and any ephemeral
datastores largely use the same data model (or schema). So config
true|false remains unchanged - the difference is the datastore you
write to.

> make the case to the owning working groups.

I do not follow here. The secure transport delivers what NETCONF calls
a username, the identity of the NETCONF client. If this client acts on
behalf of another application (the secondary identity), then this
identity is meta data should be attached to the information submitted
to the ephemeral datastore. I do not see why this would lead to any
changes to SSH or TLS.

> [Juergen, is this an action item for yang 1.1?]

The YANG 1.1 issue Y20 is about adding a set of built-in xpath
functions. I like to ask I2RS to tell us what functions they need. We
do have IP prefix-length matching on our radar. If other functions are
required, please let us know as soon as possible.

Now, this mostly is about the xpath function set that can be used in
must and when expressions. You probably want to use them also in
filtering expressions in the protocol. This is where things again
become a protocol issues. Note that RFC 6241 says on page 17:

      The XPath expression is interpreted in the following context:

      [...]

      *  The function library is the core function library.

/js

-- 
Juergen Schoenwaelder           Jacobs University Bremen gGmbH
Phone: +49 421 200 3587         Campus Ring 1, 28759 Bremen, Germany
Fax:   +49 421 200 3103         <http://www.jacobs-university.de/>

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Wed, 24 Sep 2014 09:10:24 +0200

On Tue, Sep 23, 2014 at 06:14:16PM -0400, Susan Hares wrote:
> (empheral)? 

The beauty of the fourth option is that it does not require a large
set of new data models. The idea is that you have a standard config
data model and you use the same model for an ephemeral datastore. The
server will then take the contents of the config datastore and 'merge'
it with the content of any ephemeral datastores to produce what the
box ends up doing (the operational state). The merging function will
take priorities into account should conflicts arise. Here is the
relevant figure (taken from the meeting minutes):

               +-----------------+
               |                 |
         +--- (+) ---+           |
         ^           ^           v
       +---+       +---+       +---+
       |   |       |   |       |   |
       |(1)|       |   |       |   |
       |   |       |   |       |   |
       +---+       +---+       +---+

     NC config  ephemeral    operational
     datastore  datastore      state

    (1) The complete NC config datastore is at certain synchronization
    points made persistent

    (+) Priority resolution, priorities may be per datastore or per
    user or per 'application' or even per data node

/js

---

From: Martin Bjorklund <[email-masked]>
Date: Wed, 24 Sep 2014 09:26:58 +0200 (CEST)

>       *  The function library is the core function library.

The quoted text is for the error-path.  For the filter, rfc 6241 says
(p 67):

   o  The function library is the core function library, plus any
      functions defined by the data model.

/martin

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Wed, 24 Sep 2014 09:32:27 +0200

>       functions defined by the data model.

Oops. So we are in a better shape than I thought. Good news I would
say (although this binding of protocol and data model leads to
interesting effects but I do not want to go down there).

/js

---

From: "Thomas D. Nadeau" <[email-masked]>
Date: Wed, 24 Sep 2014 07:35:47 -0400

> relevant figure (taken from the meeting minutes):

	These are precisely the qualities I and some others were thinking of when we started i2rs. The idea is quite simple, as you have said above and really needs not be complicated more.  

	--Tom

> netmod mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/netmod
>

---

From: "Thomas D. Nadeau" <[email-masked]>
Date: Wed, 24 Sep 2014 07:50:39 -0400

> filesystem semantics are much closer.

	Yes I agree. They are configuration objects.  There might be two copies of the same object across data stores not an inheritance relationship: say one in the run-time/ephemeral, one in the persistent.

> true|false remains unchanged - the difference is the datastore you

	They should use precisely the same data models with perhaps a sub-set being actually implemented at any given time in the ephemeral one given that some objects might not make sense in that context.  In the diagram you drew in the previous email I responded to, you showed this analogy very well I think.

	--Tom

---

From: Jeffrey Haas <[email-masked]>
Date: Wed, 24 Sep 2014 11:34:01 -0400

[...]
> 	These are precisely the qualities I and some others were thinking of when we started i2rs. The idea is quite simple, as you have said above and really needs not be complicated more.  

It has its own complications.

Do we permit more than one ephemeral datastore?  If so, this simplifies data
object priority when resolving the operational state.  But this also
complicates operational state integrity when you have multiple cross
references.

-- Jeff

---

From: Jeffrey Haas <[email-masked]>
Date: Wed, 24 Sep 2014 11:38:02 -0400

> filesystem semantics are much closer.

Accepted.  I'll stick to the union filesystem example alone for the future.

> true|false remains unchanged - the difference is the datastore you

I had forgotten this detail.

> to the ephemeral datastore. I do not see why this would lead to any

I tried to raise this as a question during the interim, but I think it was
lost.  How should the meta-data be transported?

> required, please let us know as soon as possible.

The primary ones I am aware of are operations on network addresses and
prefixes.  Was netmod looking for an explicit manifest of such operations?

This may have overlap in the work for the ACL module that is already a
netmod document.

>       *  The function library is the core function library.

We'll raise this point separately to netconf.

-- Jeff

---

From: Jeffrey Haas <[email-masked]>
Date: Wed, 24 Sep 2014 11:43:52 -0400

Sue,

> How would the fourth model be expressed in the yang model  - config

I believe Juergen has answered this point.

> 2)  Please define your definition of local config with an
> example., 

This corresponds to "config true" state, probably in the running datastore.
The semantics on the impact for the commit model when writable-running is
not in use haven't been fully explored.

> and 3) please give an example of the (must, when) use cases
> envisioned?

In the original proposal in my draft, when we had completely disjoint
datastores, consider the BGP I2RS ephemeral peer case.  The I2RS model would
have a container of ephemeral peers.  That container might not specify the
local Autonomous System number.  In that case, there would be a MUST
requirement for that neighbor that there exist a system-wide configured
Autonomous System number, probably in the local Config.

Currently, there is no syntax that says how to examine nodes that are in a
different datastore.  From an XPath context, the current datastore is the
document context.

In the new proposal, since the ephemeral datastore gets a shadow of the
local config, there is no need to have extensions to XPath to refer between
data stores.

However, if we end up with more than one ephemeral datastore (not
recommended for this reason), such an issue returns.

-- Jeff

---

From: "Thomas D. Nadeau" <[email-masked]>
Date: Wed, 24 Sep 2014 12:54:39 -0400

> complicates operational state integrity when you have multiple cross

	I vote for keeping this simple for starters. Lets get an implementation or two of a single ephemeral data store. This avoids having multiple priorities too; its just 1.

	--Tom

---

From: Dean Bogdanovic <[email-masked]>
Date: Wed, 24 Sep 2014 21:00:54 +0000

> 	I vote for keeping this simple for starters. Lets get an implementation or two of a single ephemeral data store. This avoids having multiple priorities too; its just 1.
We can allow multiple ephemeral data stores, but the only dependency can be with the NC config datastore. Example:

residential services
business services

each of the configuration sits in their own ephemeral datastore, but only dependency is on NC config store. 

As we are still keeping 1 to 1 overlay, not many to 1.

Dean

---

From: Andy Bierman <[email-masked]>
Date: Wed, 24 Sep 2014 14:19:23 -0700

On Wed, Sep 24, 2014 at 2:00 PM, Dean Bogdanovic <[email-masked]> wrote:

> >>> On Sep 24, 2014:3:10 AM, at 3:10 AM, Juergen Schoenwaelder <
> [email-masked]> wrote:
> >> [...]
> >>>>
> >>>>             +-----------------+
> >>>>             |                 |
> >>>>       +--- (+) ---+           |
> >>>>       ^           ^           v
> >>>>     +---+       +---+       +---+
> >>>>     |   |       |   |       |   |
> >>>>     |(1)|       |   |       |   |
> >>>>     |   |       |   |       |   |
> >>>>     +---+       +---+       +---+
> >>>>
> >>>     These are precisely the qualities I and some others were thinking
> of when we started i2rs. The idea is quite simple, as you have said above
> and really needs not be complicated more.
> >>
> >       I vote for keeping this simple for starters. Lets get an
> implementation or two of a single ephemeral data store. This avoids having
> multiple priorities too; its just 1.
> >
> >       --Tom
> We can allow multiple ephemeral data stores, but the only dependency can
> be with the NC config datastore. Example:
>
> residential services
> business services
>
>

What does this really mean?
I did not understand the details at the interim.
Does this mean each datastore has separate data models?
Does this mean each service represent separate I2RS clients?

The only real reason to have multiple ephemeral datastores
would be to create the same instances of 1 or more data nodes
in multiple datastores. (multiple concurrent overlays).

IMO this sort of complexity should be left out of the agent.

each of the configuration sits in their own ephemeral datastore, but only
> dependency is on NC config store.
>

> Dean

Andy

---

From: "Thomas D. Nadeau" <[email-masked]>
Date: Wed, 24 Sep 2014 17:51:09 -0400

Why not start with just 1?

---

From: Dean Bogdanovic <[email-masked]>
Date: Thu, 25 Sep 2014 01:42:47 +0000

Tom,

I'm in favor of starting with one, but can see use cases for more. Please keep in mind that in case for more ephemeral data stores, they will all have same schema, but different models in ephemeral stores. There will be no dependencies between ephemeral data stores.

Dean

---

From: Dean Bogdanovic <[email-masked]>
Date: Thu, 25 Sep 2014 01:48:01 +0000

On Sep 24, 2014, at 5:19 PM, Andy Bierman <[email-masked]<mailto:[email-masked]>> wrote:

On Wed, Sep 24, 2014 at 2:00 PM, Dean Bogdanovic <[email-masked]<mailto:[email-masked]>> wrote:

On Sep 24, 2014, at 12:54 PM, Thomas D. Nadeau <[email-masked]<mailto:[email-masked]>> wrote:

>
> On Sep 24, 2014:11:34 AM, at 11:34 AM, Jeffrey Haas <[email-masked]<mailto:[email-masked]>> wrote:
>
>>> On Sep 24, 2014:3:10 AM, at 3:10 AM, Juergen Schoenwaelder <[email-masked]<mailto:[email-masked]>> wrote:
>> [...]
>>>>
>>>>             +-----------------+
>>>>             |                 |
>>>>       +--- (+) ---+           |
>>>>       ^           ^           v
>>>>     +---+       +---+       +---+
>>>>     |   |       |   |       |   |
>>>>     |(1)|       |   |       |   |
>>>>     |   |       |   |       |   |
>>>>     +---+       +---+       +---+
>>>>

residential services
business services

yes, each data store has separate data models

each ephemeral datastore represents separate I2RS clients.

I view it differently. In subscriber management different data models are used for residential and business subscribers. Some service data models can be very unique for business customers. So separating them in different data stores makes sense, as they there are no overlaps with each other, only with NC config store. In this case, we can get better performance.

Dean

Andy

> [email-masked]<mailto:[email-masked]>

_______________________________________________
i2rs mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/i2rs

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Thu, 25 Sep 2014 08:24:56 +0200

> complicates operational state integrity when you have multiple cross

It complicates the merge operation - it does not affect the integrity
of the operational state. But even that may not be true, at least not
from the point of the merge algorithm. There likely is not much
difference between merging 1 or N ephemeral datastores. And at the end
of the day, if you have I2RS systems injecting conflicts, then it does
not matter whether that happens in 1 ephemeral datastore and N
ephemeral datastores. Having N ephemeral datastores had the advantage
that I can easily disable a complete ephemeral datastore by modifying
its priority while multiple I2RS systems writing concurrently into the
same scratchpad will be much more fun to deal with.

/js

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Thu, 25 Sep 2014 08:32:25 +0200

> lost.  How should the meta-data be transported?

Likely as meta-data attributes.

https://tools.ietf.org/html/draft-lhotka-netmod-yang-metadata-00

> prefixes.  Was netmod looking for an explicit manifest of such operations?

We are happy to receive input. A starting point is here.

http://tools.ietf.org/html/draft-bjorklund-netmod-yang-xpath-extensions-00

This I-D, however, does not define functions for network addresses.

> This may have overlap in the work for the ACL module that is already a

I am lost here. I do not see how additional xpath functions related to
the NACM (which came out of NETCONF not NETMOD).

/js

---

From: "Joel M. Halpern" <[email-masked]>
Date: Thu, 25 Sep 2014 06:12:00 -0400

Separate ephemeral data stores for distinct I2RS clients would seem a 
very bad idea.
It is perfectly acceptable for one client to read information being 
manipulated by another cleitn.  And if so, the reader should see the 
results of that other clients activity.

Also, separate stores would not produce the error results required in 
the case of collision.

So no, I do not see a use case from I2RS for multiple ephemeral data stores.

Yours,
Joel

On 9/24/14, 9:48 PM, Dean Bogdanovic wrote:
>>     On Sep 24, 2014, at 12:54 PM, Thomas D. Nadeau
>>     <[email-masked] <mailto:[email-masked]>> wrote:
>>
>>     >
>>     > On Sep 24, 2014:11:34 AM, at 11:34 AM, Jeffrey Haas
>>     <[email-masked] <mailto:[email-masked]>> wrote:
>>     >
>>     >>> On Sep 24, 2014:3:10 AM, at 3:10 AM, Juergen Schoenwaelder
>>     <[email-masked]
>>     <mailto:[email-masked]>> wrote:
>>     >> [...]
>>     >>>>
>>     >>>>             +-----------------+
>>     >>>>             |                 |
>>     >>>>       +--- (+) ---+           |
>>     >>>>       ^           ^           v
>>     >>>>     +---+       +---+       +---+
>>     >>>>     |   |       |   |       |   |
>>     >>>>     |(1)|       |   |       |   |
>>     >>>>     |   |       |   |       |   |
>>     >>>>     +---+       +---+       +---+
>>     >>>>
>>     >>>     These are precisely the qualities I and some others were
>>     thinking of when we started i2rs. The idea is quite simple, as you
>>     have said above and really needs not be complicated more.
>>     >>
>>     implementation or two of a single ephemeral data store. This
>>     avoids having multiple priorities too; its just 1.
>>     >
>>     >       --Tom
>>     We can allow multiple ephemeral data stores, but the only
>>     dependency can be with the NC config datastore. Example:
>>
>>     residential services
>>     business services
>>
>>
>>
> I view it differently. In subscriber management different data models
> are used for residential and business subscribers. Some service data
> models can be very unique for business customers. So separating them in
> different data stores makes sense, as they there are no overlaps with
> each other, only with NC config store. In this case, we can get better
> performance.
>>
>>     each of the configuration sits in their own ephemeral datastore,
>>     but only dependency is on NC config store.
>>
>>     > [email-masked] <mailto:[email-masked]>
>>     [email-masked] <mailto:[email-masked]>

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Thu, 25 Sep 2014 12:26:40 +0200

Joel,

my expectations about the intelligence of I2RS clients is perhaps
lower than yours. Do you really believe I2RS clients will be smart to
adapt to whatever some other unknown conflicting I2RS client injects
into a shared ephemeral datastore?

I do not expect smartness. My hope is that I2RS clients involved in a
conflict can be signalled and probably more important that a human
operator can be signalled that I2RS clients step into each other so
that a human can take action to resolve things. Automatic conflict
resultion based on observation what state is injected by other I2RS
clients seems to me a huge (idealistic) assumption.

I would find conflict detection during the merge operation likely 
more reliable than conflict detection based on smart I2RS clients
monitoring closely what is going on in a shared datastore.

I assume there will be dumb I2RS clients.

/js

---

From: Martin Bjorklund <[email-masked]>
Date: Thu, 25 Sep 2014 12:33:13 +0200 (CEST)

> lost.  How should the meta-data be transported?

In general, NETCONF / RESTCONF with YANG supports the concept of
meta-data, and meta-data is in XML transported as an XML attribute.
(For JSON encoding see
http://tools.ietf.org/id/draft-lhotka-netmod-yang-metadata-00.txt)

> prefixes.  Was netmod looking for an explicit manifest of such operations?

Yes this would help.

/martin

---

From: Martin Bjorklund <[email-masked]>
Date: Thu, 25 Sep 2014 12:39:30 +0200 (CEST)

Dean Bogdanovic <[email-masked]> wrote:
> 
> On Sep 24, 2014, at 5:19 PM, Andy Bierman
> <[email-masked]<mailto:[email-masked]>> wrote:
> We can allow multiple ephemeral data stores, but the only dependency
> can be with the NC config datastore. Example:
> 
> residential services
> business services
> 
> 
> 

Now I am confused.  At the interim we agreed (and you argued for :)
that the schema (= data models) is the same for the config and
ephemeral datastore.

> models can be very unique for business customers.

Ok.

> So separating them
> in different data stores makes sense, as they there are no overlaps
> with each other, only with NC config store.

But since they are disjoint, there are really no reasons for not
keeping them in the same datastore, since there will be no conflicts.

> In this case, we can get
> better performance.

I can't see why, but this is probably an implementation issue.

/martin

---

From: "Joel M. Halpern" <[email-masked]>
Date: Thu, 25 Sep 2014 07:14:21 -0400

(I believe that your talking about dumb clients is because I2RS and 
NetCONF YANG use terms differently.  In I2RS, the Agent is the thing in 
the box, and the Client is the smarter things outside the box.)

We wrote very specific rules into the architecture document the WG 
adopted, specifically because we wanted to bound the expected 
intelligence of the I2RS Agent.
The agent is required to detect collision.  It is required to generate 
error notification upon collision.
It is also required to use the priority mechanism to produce a 
predictable composite result in the case of collision.

The odds of that result being what the human beings want is pretty small 
  But it is predictable.  Which is all we felt we could ask for.

Yours,
Joel

---

From: "Thomas D. Nadeau" <[email-masked]>
Date: Thu, 25 Sep 2014 08:19:06 -0400

On Sep 25, 2014:6:12 AM, at 6:12 AM, Joel M. Halpern <[email-masked]> wrote:

> Separate ephemeral data stores for distinct I2RS clients would seem a very bad idea.
> It is perfectly acceptable for one client to read information being manipulated by another cleitn.  And if so, the reader should see the results of that other clients activity.

	I agree with Joel here. I was quite surprised by Dean's first note making this statement. This seems like undue complication to the whole situation. Making this work can be quite straight-forward.   The problem with having a store per-client is that well, clients can be ephemeral too. People need to remember that the i2rs clients are probably going to be applications that might come and go, and as such should a device have to keep around a lot of state for everyone that talks to it via its Rest/i2rs API?  That does not sound like a good idea to me, even if we time it out.

> Also, separate stores would not produce the error results required in the case of collision.

	Yes, that is precisely another issue that came to mind - besides the scale of having one per client, do we really want to place the burden on the device to let all clients know of collisions when merging?  What if those clients have "gone away"? 

> So no, I do not see a use case from I2RS for multiple ephemeral data stores.

	Agree %100.

	--Tom

---

From: "Thomas D. Nadeau" <[email-masked]>
Date: Thu, 25 Sep 2014 08:29:57 -0400

> I assume there will be dumb I2RS clients.

	I too assume "dumb" clients (i.e.: or those with minimal computational horsepower). Given that assumption, I've thought that the "merge" operation should be a simple copy-over-the-stored-config operation. This is why we shouldn't have more than one ephemeral store. If its treated this way, its quite literally, just a copy-over operation.  My original ideas around i2rs were always such that the ephemeral store was just a scratch pad where objects were stored to reflect a shadow of a subset of the running config (just the forwarding state information) until such time as it could get copied there "soon" after, either via a timer popping or an explicit command from the OS or client(s).  Access to that store was done using a variant of Netconf that a) avoided the configuration verification process from Netconf and b) was RESTful in nature. What we have been discussing here with multiple stores (and priorities) has vastly and IMHO unnecessarily, complicated the original idea which was quite simple and obtainable with mostly off-the-shelf technological components (i.e.: RESTConf + a hacked Netconf daemon + yang models).  It is the latter recipe from with which I think we should use as a starting point to build crash test dummy implementations from to see if this thing works rather than discussing the theoretical aspects of complicating that further.  Sorry, just my $.02. 

	--Tom

---

From: Dean Bogdanovic <[email-masked]>
Date: Thu, 25 Sep 2014 12:36:45 +0000

> that the schema (= data models) is the same for the config and
Here is an example. The device supports x number of features and has a data model that represents the whole device. From that data model (schema), I decide to build a service that is using only parts of all available and store it into ephemeral DB, example L3VPN. Now in ephemeral only L3VPN service configurations will be stored.

---

From: "Thomas D. Nadeau" <[email-masked]>
Date: Thu, 25 Sep 2014 08:44:01 -0400

> Here is an example. The device supports x number of features and has a data model that represents the whole device. From that data model (schema), I decide to build a service that is using only par…

	I2RS does not model the entire device; just its "routing system" components.

	--Tom

---

From: Martin Bjorklund <[email-masked]>
Date: Thu, 25 Sep 2014 14:44:31 +0200 (CEST)

Dean Bogdanovic <[email-masked]> wrote:
> Here is an example. The device supports x number of features and has a
> data model that represents the whole device. From that data model
> (schema), I decide to build a service that is using only parts of all
> available and store it into ephemeral DB, example L3VPN. Now in
> ephemeral only L3VPN service configurations will be stored.

This is fine.  This means that the *schema* is the same for both data
stores, but the instance data is different.

Note that the set of data models make up the schema; thus both data
stores have the *same* data models (same schema), not separate.

/martin

---

From: Jeffrey Haas <[email-masked]>
Date: Thu, 25 Sep 2014 10:23:01 -0400

[...]
> I too assume "dumb" clients (i.e.: or those with minimal computational
> horsepower). Given that assumption, I've thought that the "merge"
> operation should be a simple copy-over-the-stored-config operation.

Mostly correct, but with the idea that you can "delete" from the ephemeral
datastore perspective shadowed local config state by effectively leaving a
hole in the ephemeral config.  E.g.:

Local Config:
A[1]
A[2]
A[3]

Ephemeral Config:
DELETE A[2]
A[4]

Active Config (post merge):
A[1]
A[3]
A[4]

-- Jeff

---

From: Jeffrey Haas <[email-masked]>
Date: Thu, 25 Sep 2014 10:25:37 -0400

> 	I2RS does not model the entire device; just its "routing system" components.

While we're primarily concerned with I2RS semantics, the discussion has
effectively broadened because the ephemeral mechanism we're discussing may
be used by non-I2RS clients.

I chatted with Dean a few minutes ago.  I think much of the confusion
regarding multiple ephemeral datastores is a result of a JUNOS
implementation detail.  He'll clarify the externally visible behavior which
is more like what we've previously disucssed.

-- Jeff

---

From: Jeffrey Haas <[email-masked]>
Date: Thu, 25 Sep 2014 10:27:56 -0400

> https://tools.ietf.org/html/draft-lhotka-netmod-yang-metadata-00

More homework for me. Thanks.

> the NACM (which came out of NETCONF not NETMOD).

I was referring to
http://tools.ietf.org/html/draft-bogdanovic-netmod-acl-model-00

-- Jeff

---

From: "Susan Hares" <[email-masked]>
Date: Thu, 25 Sep 2014 18:25:51 -0400

Juregen:

Option 4 does make things easy.  What happens if a particular node/variables
that I2RS wants, would not normally be in the configuration?  Should you
configure that node in the empheral store only? 

Sue 

-----Original Message-----
From: i2rs [mailto:[email-masked]] On Behalf Of Juergen Schoenwaelder
Sent: Wednesday, September 24, 2014 3:10 AM
To: Susan Hares
Cc: 'Jeffrey Haas'; [email-masked]; [email-masked]
Subject: Re: [i2rs] [netmod] Summary of discussion from netmod interim on
i2rs requirements on netmod/netconf

> The fourth option is interesting and creative. A few 

> questions/requests: 1) How would the fourth model be expressed in the 

> yang model  - config (empheral)?

The beauty of the fourth option is that it does not require a large set of
new data models. The idea is that you have a standard config data model and
you use the same model for an ephemeral datastore. The server will then take
the contents of the config datastore and 'merge'

it with the content of any ephemeral datastores to produce what the box ends
up doing (the operational state). The merging function will take priorities
into account should conflicts arise. Here is the relevant figure (taken from
the meeting minutes):

               +-----------------+

               |                 |

         +--- (+) ---+           |

         ^           ^           v

       +---+       +---+       +---+

       |   |       |   |       |   |

       |(1)|       |   |       |   |

       |   |       |   |       |   |

       +---+       +---+       +---+

    points made persistent

/js

-- 

Fax:   +49 421 200 3103         < <http://www.jacobs-university.de/>
http://www.jacobs-university.de/>

i2rs mailing list

 <mailto:[email-masked]> [email-masked]

 <https://www.ietf.org/mailman/listinfo/i2rs>
https://www.ietf.org/mailman/listinfo/i2rs

---

From: "Susan Hares" <[email-masked]>
Date: Thu, 25 Sep 2014 19:01:15 -0400

Martin:

Does this mean if the I2RS model provides additional functions that are not
a base configuration model that the xpath should just work? 

Xpath1- Config - ISIS Yang model that configures 
Xpath2 - I2RS ISIS  empheral - inherits all the configuration state 
____       I2RS ISIS actions -     report on the interface changes on ISIS
L1 interfaces

Is my I2RS ISIS model a combination of the inherited state and the I2RS
actions? 

Sue 

-----Original Message-----
From: i2rs [mailto:[email-masked]] On Behalf Of Martin Bjorklund
Sent: Wednesday, September 24, 2014 3:27 AM
To: [email-masked]
Cc: [email-masked]; [email-masked]; [email-masked]
Subject: Re: [i2rs] [netmod] Summary of discussion from netmod interim on

Juergen Schoenwaelder <[email-masked]> wrote:
> > Some discussion was given to the filtering considerations.  
> > Extending the filtering options of the ietf-inet-types module may be
appropriate.
>       *  The function library is the core function library.

The quoted text is for the error-path.  For the filter, rfc 6241 says (p
67):

/martin

---

From: Martin Bjorklund <[email-masked]>
Date: Fri, 26 Sep 2014 08:43:59 +0200 (CEST)

> a base configuration model that the xpath should just work? 

It means that the protocol specification does not lock down the set of
XPath functions available in a filter.

However, there is no standard way currently to declare new XPath
functions in YANG data models.  (see the expired
https://tools.ietf.org/html/draft-bjorklund-netmod-yang-xpath-extensions-00
for some ideas in this area).

(For the set of core XPath functions see http://www.w3.org/TR/xpath/#corelib)

/martin

---

From: "Susan Hares" <[email-masked]>
Date: Fri, 26 Sep 2014 12:20:15 -0400

Juregen and Jeff:

Can you define what a merge is?  Could you define it using the example below
and another one? I know you have talked around the subject, but I need
another examples so I can adjust the drafts in progress with co-authors
(ISIS IM-DM, OSPF IM-DM, BGP IM-DM, PBR IM/DM) for i2rs.    

Sue 

============
  list foo {
    key id;
    leaf id { type int32; }
    leaf a { type int32; }
  }

local config:

   foo 42

In ephemeral config we now do SET /foo[id=42]/a  to 4711.  Thus, in
ephemeral we now have a single node (a) with value 4711.

-----Original Message-----
From: Juergen Schoenwaelder [mailto:[email-masked]] 
Sent: Thursday, September 25, 2014 2:25 AM
To: Jeffrey Haas
Cc: Thomas D. Nadeau; [email-masked]; Susan Hares; [email-masked]
Subject: Re: [i2rs] [netmod] Summary of discussion from netmod interim on

<[email-masked]> wrote:
> > 	These are precisely the qualities I and some others were thinking of
when we started i2rs. The idea is quite simple, as you have said above and
really needs not be complicated more.  
> Do we permit more than one ephemeral datastore?  If so, this 
> simplifies data object priority when resolving the operational state.  
> But this also complicates operational state integrity when you have 
> multiple cross references.

It complicates the merge operation - it does not affect the integrity of the
operational state. But even that may not be true, at least not from the
point of the merge algorithm. There likely is not much difference between
merging 1 or N ephemeral datastores. And at the end of the day, if you have
I2RS systems injecting conflicts, then it does not matter whether that
happens in 1 ephemeral datastore and N ephemeral datastores. Having N
ephemeral datastores had the advantage that I can easily disable a complete
ephemeral datastore by modifying its priority while multiple I2RS systems
writing concurrently into the same scratchpad will be much more fun to deal
with.

/js

---

From: "Susan Hares" <[email-masked]>
Date: Fri, 26 Sep 2014 12:22:25 -0400

Jeff:

Just to confirm three facts before I put them in i2rs drafts (based on
Juergen's email

- based on Juergen's email:   I2RS is "config true empheral".  

If i2rs has additional configuration for BGP related to your I2RS peers for
reporting routes: 

.         BGP based config + I2RS additional config + AS Config

Can I2RS have additional configuration values specific to I2RS?  For
example, information regarding how often to report the routes for a
particular pear?  

Sue 

-----Original Message-----
From: Jeffrey Haas [mailto:[email-masked]] 
Sent: Wednesday, September 24, 2014 11:44 AM
To: Susan Hares
Cc: 'Jeffrey Haas'; [email-masked]; [email-masked]
Subject: Re: [i2rs] Summary of discussion from netmod interim on i2rs

Sue,

> 2)  Please define your definition of local config with an example.,

-- Jeff

---

From: "Susan Hares" <[email-masked]>
Date: Fri, 26 Sep 2014 13:11:02 -0400

Jeff:

Could you give me an example of the interim's thoughts on commit versus
Alexander's message: 

http://www.ietf.org/mail-archive/web/i2rs/current/msg01974.html

Sue 

-----Original Message-----
From: i2rs [mailto:[email-masked]] On Behalf Of Jeffrey Haas
Sent: Wednesday, September 24, 2014 11:44 AM
To: Susan Hares
Cc: 'Jeffrey Haas'; [email-masked]; [email-masked]
Subject: Re: [i2rs] Summary of discussion from netmod interim on i2rs

Sue,

-- Jeff

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Mon, 29 Sep 2014 08:31:49 +0200

> (ISIS IM-DM, OSPF IM-DM, BGP IM-DM, PBR IM/DM) for i2rs.    

Merge as in NETCONF. We also discussed to have meta data attached to
nodes in ephemeral datastores in order to allow things to be deleted
etc.

/js

---

From: Jeffrey Haas <[email-masked]>
Date: Mon, 29 Sep 2014 10:15:22 -0400

[Note that I'm responding to things in thread-order.  I2RS was very busy for
a few days while I had to pay attention to other things, so my responses may
appear to jump around a bit.]

Sue,

> - based on Juergen's email:   I2RS is "config true empheral".  

This is a point of discussion.

A critical thing to realize from the netmod interim is that it was simply a
first step to trying to define what "ephemeral" even *means* in the context
of existing yang and netconf/restconf.  The "option 4" solution was the
proposal that worked its way through discussion points with the yang
experts, but still needs discussion within I2RS and then related followup
with netconf if we decide it's the way to go.

I like the idea, but as discussion is pointing out, there's a lot of details
to reconcile.

The impact of this upon writing data models for I2RS would appear to be at
this point:
- They're just "config true" (per correction from Juergen), but simply done
  in the ephemeral datastore.  How you document that they're just in the
  ephemeral data store is unclear at this point.
- They're otherwise just normal models.
- But if they're intended to be i2rs ephemeral extensions on local config
  models (e.g. protocol WG data models), we have some dependency work to
  reconcile with those groups.

> example, information regarding how often to report the routes for a

Correct.  

We have roughly four scenarios with regard to modeling impact:
A. Completely disjoint models for I2RS that use no state from other protocol
   modules and no dependencies, but interact with underlying state of that
   protocol.  E.g. BGP configuration in an I2RS model that doesn't extend a IDR
   Yang module nor have constraints in such a module.
B. A disjoint model for I2RS that has constraints against other modules.
   E.g. the I2RS BGP module may have a "must" statement against a IDR yang
   module's "autonomous system" configuration.
C. A hybrid I2RS module that augments an underlying protocol module.  This
   is the scenario you're suggesting above.  The critical detail is since it's
   an augmentation it is suggesting there exists such a protocol module.
D. There is a unified module developed in some working group that may simply
   choose to store some of its state in an ephemeral fashion.

D is the newest option enabled by the "option 4" discussion from the netmod
interim.

-- Jeff

---

From: Jeffrey Haas <[email-masked]>
Date: Mon, 29 Sep 2014 10:16:36 -0400

Sue,

> http://www.ietf.org/mail-archive/web/i2rs/current/msg01974.html

I'll address this point in the original thread.

-- Jeff