Subject: [i2rs] Conversation on Priority and Panes

From: "Russ White" <[email-masked]>
Date: Mon, 2 Nov 2015 19:39:28 -0500

Y'all --

After sleeping on the discussion last night, I think the panes of glass (or
is it pains of glass?? :-)  ) is still by and large another expression of
the priority concept within I2RS. The concept does bring up one specific
point of interest, however -- what about backup information? Some vendor
RIBs, for instance, allow a routing process to install not only the best
path as calculated by the process -- but if the process fails to install,
some RIB implementations allow the process to place the route in the "backup
pool." This allows the local RIB process to drop to the "second best path,"
in terms of priority, so the local RIB doesn't need to query the routing
processes to switch in the case of a withdraw or change in topology.

To convert this to I2RS terms, it does seem worthwhile to me to have the
capability for a local agent to accept an install instruction for some
particular ephemeral state, and if the install fails, to hold that state for
future use. This would apply to any sort of ephemeral data, including that
which is configured locally on the network device. Rather than trying to
think of this as "panes of glass," though, this would convert it to simply a
backup list of lower priority items the local agent can use in the case of
the failure of the highest priority item currently in use.

The nice thing about this view is that it doesn't require a lot of changes
at the protocol level. The only thing that needs to be available is the
option for an agent to send three different types of answers to an install
request --

1. This ephemeral state was installed and is now being used.
2. This ephemeral state was rejected/not installed -- with potential codes
for why (out of range parameter, etc.)
3. This ephemeral state was not installed, but is being held as a backup.

Using these semantics, the actual implementation of such a feature is up to
the local agent. It might be that some agents don't know how to hold backup
information, or that it doesn't make any sense to hold some sorts of
information in a backup list. This is fine -- the install can just be
rejected without further note. Locally configured information could simply
be treated as an item on the backup list, such that the priorities can be
considered as always in deciding what to install when any particular action
is taken.

It seems, to me, that this is a simpler way to consider the same problem
set, and reduces to an actual protocol mechanism that appears (?) to be
fairly simple, and leaves as much flexibility as possible for any given
agent implementation.

Thoughts?

:-)

Russ

---

From: "Joel M. Halpern" <[email-masked]>
Date: Mon, 2 Nov 2015 20:08:51 -0500

We discussed storing backup ephemeral state.  There are a number of 
issues that arise if we permit that.

For example, if state gets over-ridden, but preserved, then even though 
it is not doing any good, the client which installed the state must 
still monitor for any related dependenecies so as to not leave incorrect 
pending state.
Which also means that a client has to be able to remove state it has 
created, even though that state has been over-ridden.  And probably has 
to be able to create state which won't take effect.  Which in turns 
means that you need to be able to read your own effects and the current 
state separately.

In general, earlier working group discussion found this to add a lot of 
complexity.  So our agreement for now is that there is no storing of 
non-effecting state by I2RS.  Caching or other performance improvements 
are for future study.

Yours,
Joel

On 11/2/15 7:39 PM, Russ White wrote:
> _______________________________________________
> i2rs mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/i2rs
>

---

From: "Russ White" <[email-masked]>
Date: Mon, 2 Nov 2015 20:28:22 -0500

> For example, if state gets over-ridden, but preserved, then even though it
is
> not doing any good, the client which installed the state must still
monitor for
> any related dependenecies so as to not leave incorrect pending state.
created,
> even though that state has been over-ridden.  And probably has to be able
> to create state which won't take effect.  Which in turns means that you
need
> to be able to read your own effects and the current state separately.

I would think this should be a client specific implementation detail... I
don't see why, from a protocol level, it should be disallowed, when the
changes involved would be minor (or at least it seems to me?).

:-)

Russ

---

From: "Joel M. Halpern" <[email-masked]>
Date: Mon, 2 Nov 2015 20:51:29 -0500

Allowing caching means that we have to specify additional mechanisms 
(such as read-through and write-through, and returns for successful 
writes that do not actually take effect, and probably other aspects.)

So we agreed that was for future consideration, as it is by no means minor.

Yours,
Joel

---

From: Andy Bierman <[email-masked]>
Date: Mon, 2 Nov 2015 18:02:18 -0800

On Mon, Nov 2, 2015 at 5:28 PM, Russ White <[email-masked]> wrote:

> it
I am not sure I understand edit collision processing.
The proposals is that the ephemeral datastore contains data
from 1 or more clients.  The data from different clients is non-overlapping
(for some definition of that).

I will avoid running config + ephemeral leaf-override for now.
Assume the data model is entirely for I2RS:

  container foo {
     leaf A { type int32; }
     leaf B { type int32; }
  }

1) client-worst set /foo/A to 42    { "foo": { "A":42 } }

2) client-best sets /foo/B to 7       { "foo": { "B":7 } }

3) Is client-best going to knock out data owned by client-worst?
   what happens to /foo/A?

Q1) Is this a collision because /foo is written by both clients?
  if not, does client-best own /foo and /foo/A, and client-worst owns
/foo/B?
  If yes, then step (2) is effectively a PUT, no matter what method is
really used

Q2) what notifications are sent to client-worst, if any (depending on
answer to Q1)
(client-worst either lost ownership of /foo or /foo and /foo/A)

Consider that A and B can be arbitrary subtrees, not just leafs.

How are the edit collision boundaries specified?
Simple YANG overlap will cause lots of deletions.
More granular/complex boundary specification would help
avoid such deletions where the 2 clients could actually co-exist.

> Russ

Andy

---

From: Joel Halpern Direct <[email-masked]>
Date: Mon, 2 Nov 2015 21:05:27 -0500

What we side with regard to the atomicity of collisions was that the 
model needed to specify that.  So if the model writer concludes that the 
leaves A and B in your example are so closely coupled that bad things 
will happen if different people write different parts, then they can say 
the container is the granularity.  Usually, I would expect the leaf to 
be the granularity.  We do not expect the I2RS agent (or the underlying 
system) to guess the granularity.

Yours,
Joel

On 11/2/15 9:02 PM, Andy Bierman wrote:
>
>
> On Mon, Nov 2, 2015 at 5:28 PM, Russ White <[email-masked]
> <mailto:[email-masked]>> wrote:
>
>
>     [email-masked] <mailto:[email-masked]>

---

From: "Russ White" <[email-masked]>
Date: Mon, 2 Nov 2015 21:12:40 -0500

> Allowing caching means that we have to specify additional mechanisms (such
> as read-through and write-through, and returns for successful writes that
do
> not actually take effect, and probably other aspects.)

I don't really see it as "caching..." I'm thinking more of the backup route
situation in the RIB, specifically.

> So we agreed that was for future consideration, as it is by no means
minor.

I would argue it's worthwhile to at least leave space in the protocol
definition for the third return option, and leave it up to agent
implementations to deal with the complexity if desired in the future.

:-)

Russ

---

From: Andy Bierman <[email-masked]>
Date: Mon, 2 Nov 2015 18:16:11 -0800

On Mon, Nov 2, 2015 at 6:05 PM, Joel Halpern Direct <
[email-masked]> wrote:

> What we side with regard to the atomicity of collisions was that the model
> needed to specify that.  So if the model writer concludes that the leaves A
> and B in your example are so closely coupled that bad things will happen if
> different people write different parts, then they can say the container is
> the granularity.  Usually, I would expect the leaf to be the granularity.
> We do not expect the I2RS agent (or the underlying system) to guess the
> granularity.
>
>
OK -- so it is part of the data model definition.  Good.
These are the details wrt/ "first wins" that need to be understood.
The WG and proto-dt needs to think about what YANG extensions would be
needed
to describe these boundaries in machine-usable form.

Yours,
Andy

---

From: Andy Bierman <[email-masked]>
Date: Mon, 2 Nov 2015 20:12:27 -0800

> We do not expect the I2RS agent (or the underlying system) to guess the

It is 1 thing to just declare this feature in the architecture and quite
another
to get it to work with real YANG.  In fact, there is no 1 "data model".
There is an ever-growing set of modules, with various forms and degrees
of inter-dependence.

Dean's draft does a good job of explaining these module types:
https://tools.ietf.org/html/draft-bogdanovic-netmod-yang-model-classification-05

We cannot just assume that data nodes in different modules do not interact
and cause a conflict.  Quite the opposite, since a common data modeling
practice is to create a framework and a set of augmenting modules for
that framework.

It is OK if there are different levels of interoperability expectations.
Some details can be mandatory, some left for future work,
some specifically left to the server implementation.
This level of detail is still very TBD.

Yours,

Andy

---

From: "Susan Hares" <[email-masked]>
Date: Mon, 2 Nov 2015 23:32:18 -0500

Russ thank you for kicking off this discussion.  It is an interesting
approach. I know that certain RIB implementations allows a back-up route. 

Sue 

-----Original Message-----
From: i2rs [mailto:[email-masked]] On Behalf Of Russ White
Sent: Monday, November 02, 2015 7:39 PM
To: [email-masked]
Subject: [i2rs] Conversation on Priority and Panes

Y'all --

1. This ephemeral state was installed and is now being used.
2. This ephemeral state was rejected/not installed -- with potential codes
for why (out of range parameter, etc.) 3. This ephemeral state was not
installed, but is being held as a backup.

Thoughts?

:-)

Russ

---

From: Andy Bierman <[email-masked]>
Date: Tue, 3 Nov 2015 15:14:48 -0800

Hi,

This raises a data modeling issue.
Should every "backup parameter" be modeled explicitly in
the YANG module, or can the ephemeral datastore be used
for that, without any additional data model objects?

The I2RS architecture supports this "backup" concept (lower priority
client),
except it requires a notification from the agent and subsequent request
from the client to make the backup active.  With NETCONF or RESTCONF
today, that round-trip will probably take around  500 to 1000 milli-seconds.
Probably much worse during high loads.

Our original proposal to the design team included a pane of glass
for each client (and unique priorities for each client), because some people
were talking about sub-milli-sec latency, and I know there is no way NETCONF
or RESTCONF is going to support this sort of tight control loop.

Whether the server rejects lower-priority edits right away, or whether
the agent caches the request in the form of a client-specific pane,
the client still needs to observe the data resources with pub/sub
and decide whether its own particular state is still relevant.
IMO the client complexity is the same either way, especially
since the caching is only done if the client requests it.

The only difference is likely to be almost a million times faster
fail-over to the backup on the server.

Andy

---

From: "Joel M. Halpern" <[email-masked]>
Date: Tue, 3 Nov 2015 18:49:07 -0500

The basic problem I have with your description is that it treats 
over-writes as normal and desirable, and assumes that the priority 
handling is producing the correct results.  If we actually believed 
that, I suppose making them more efficient would be sensible.

But that is not actually what we are doing.  Priority over-write is a 
disambiguation mechanism.  There is no expectation that it is even a 
good heuristic for correctness.  It is merely predictable.  Trying to 
optimize the control loop for cases of improper behavior seems the wrong 
place to optimize.

Having said that, if we want to get into multiple panes of ephemeral 
glass then we are going to need mechanisms to
read the composite effect
read what I as a client have applied
indicate in the response to a write request that the agent has accepted 
the request, even though it is not actually taking effect.

And if we do all that, clients whose state is pending will need to 
maintain monitoring of all related activities even though their network 
application is not in effect.

And, if there are multiple aspect of an operation, if one gets 
over-written but retained, then the client probably can't leave it 
there, but has to go in and remove that state, since the client will 
likely be removing the rest of the related state that is still sitting 
there with its lynchpin missing.

And then we get into the question of how much unapplied state is an 
agent going to store.  So it all probability the client still has to be 
prepared for being told that not only was its state over-written (which 
is technically an error) but that it got deleted too.

Yours,
Joel

On 11/3/15 6:14 PM, Andy Bierman wrote:
> On Mon, Nov 2, 2015 at 8:32 PM, Susan Hares <[email-masked]
> <mailto:[email-masked]>> wrote:
>
>     From: i2rs [mailto:[email-masked]
>     <mailto:[email-masked]>] On Behalf Of Russ White
>     To: [email-masked] <mailto:[email-masked]>

---

From: "Russ White" <[email-masked]>
Date: Wed, 4 Nov 2015 01:57:44 -0500

> The basic problem I have with your description is that it treats
over-writes as
> normal and desirable, and assumes that the priority handling is producing
> the correct results.  If we actually believed that, I suppose making them
> more efficient would be sensible.

After talking to Jeff a bit, I actually think we're probably trying to solve
two different sets of problems in the same set of documents... The original
idea of a "tight loop to the RIB" is sharing space here with a longer term,
more static (but ephemeral) idea. It might be better if we actually split
these two sets of requirements up.

In the "tight loop to the RIB" case (represented by every use case I
original put in the doc's I originally presented at the beginning of the
WG), I'm pretty certain overwrites by multiple clients with callbacks in a
very short time period are going to be a normal case -- two processes
installing the same prefix in the RIB is a common situation in normal
routing, and, in fact, something I would expect. Think of the case where a
flow or destination needs to be redirected for some short period of time,
and then traffic rerouted back to the pre-existing path, particularly in
cases like software defined perimeter, or putting the user on a dead end
network so they can only reach an authentication server, or redirecting a
flow temporarily for traffic engineering purposes -- all of these could
include entries that last under a second in the table.

The "ephemeral configuration" situation is more akin to a static route -- if
there are two, there's probably some error (perhaps). In reality, I've
configured many pairs of static routes over the years such that if the first
next hop is pulled from the table, the second next hop should be
_immediately_ installed -- there's no time to wait for a round trip to the
controller, so I don't know if I accept the premise that two static routes
installed by two different controllers is a misconfiguration on its face.

It might be good to separate these two use cases and their attendant
solutions, rather than burying the "tight loop to the RIB" under more
complex mechanisms designed for the apparent "long term ephemeral state"
solution.

:-)

Russ

---

From: "Susan Hares" <[email-masked]>
Date: Wed, 4 Nov 2015 17:23:42 -0500

Joel: 

On the reading of composite panes, if you believe that ephemeral can be set
on any configuration node or as an independent then the following 

Config-node-1 
	 ephemeral node-1 (client 1), 
              ephemeral Node-1 (client 2), 

Each ephemeral node could have an ID and a priority.  The composite mode can
apply a consistent policy:  (E.g. high priority with tagging for
first-wins).   Asking for a composite in the rpc for read is a possible use
for the composite.  Asking for all of these nodes is possible. 

I believe this caching is required for the "tight-loop" issue of < 1 second
response for query/response. 

If we define the group issues in the Yang language, then I think we can
handle the multiple I2RS ephemeral entries with more memory space. The
amount of memory space used by the I2RS caching entries can be set by the
implementation in the Agent, and expressed by the model capabilities.  I
agree with Andy's original position that Caching will be necessary for high
performance on medium to large scale Data models. 

Lastly, I am not sure our consensus on the "no-caching" was strong enough to
refuse to consider it now.  Meaning less than 10 people in interims or email
-- should not prevent a larger group from reconsidering.  This is a
different type of consensus as a long debate on list plus IETF discussions.

Sue 
-----Original Message-----
From: i2rs [mailto:[email-masked]] On Behalf Of Joel M. Halpern
Sent: Tuesday, November 03, 2015 6:49 PM
To: Andy Bierman; Susan Hares
Cc: [email-masked]; Russ White
Subject: Re: [i2rs] Conversation on Priority and Panes

The basic problem I have with your description is that it treats over-writes
as normal and desirable, and assumes that the priority handling is producing
the correct results.  If we actually believed that, I suppose making them
more efficient would be sensible.

But that is not actually what we are doing.  Priority over-write is a
disambiguation mechanism.  There is no expectation that it is even a good
heuristic for correctness.  It is merely predictable.  Trying to optimize
the control loop for cases of improper behavior seems the wrong place to
optimize.

Having said that, if we want to get into multiple panes of ephemeral glass
then we are going to need mechanisms to read the composite effect read what
I as a client have applied indicate in the response to a write request that
the agent has accepted the request, even though it is not actually taking
effect.

And if we do all that, clients whose state is pending will need to maintain
monitoring of all related activities even though their network application
is not in effect.

And, if there are multiple aspect of an operation, if one gets over-written
but retained, then the client probably can't leave it there, but has to go
in and remove that state, since the client will likely be removing the rest
of the related state that is still sitting there with its lynchpin missing.

And then we get into the question of how much unapplied state is an agent
going to store.  So it all probability the client still has to be prepared
for being told that not only was its state over-written (which is
technically an error) but that it got deleted too.

Yours,
Joel

On 11/3/15 6:14 PM, Andy Bierman wrote:
> Should every "backup parameter" be modeled explicitly in the YANG 
> module, or can the ephemeral datastore be used for that, without any 
> additional data model objects?
>
> client), except it requires a notification from the agent and 
> subsequent request from the client to make the backup active.  With 
> NETCONF or RESTCONF today, that round-trip will probably take around  
> 500 to 1000 milli-seconds.
> Our original proposal to the design team included a pane of glass for 
> each client (and unique priorities for each client), because some 
> people were talking about sub-milli-sec latency, and I know there is 
> no way NETCONF or RESTCONF is going to support this sort of tight control
loop.
> the client still needs to observe the data resources with pub/sub and 
> decide whether its own particular state is still relevant.
> IMO the client complexity is the same either way, especially since the 
> caching is only done if the client requests it.
>

---

From: Joel Halpern Direct <[email-masked]>
Date: Wed, 4 Nov 2015 18:41:30 -0500

The working group can always reconsider any decision.
But if we are going to do so, we need to recognize that the earlier 
discussion covered a LOT of issues that need to be dealt with.

I do not understand the question in the first part of your email, so I 
can not comment on it.

It is quite clear that caching is not required to meet a <1 second loop. 
  In particular, this seems to be a major change in another agreement. 
We as a group concluded that over-write and other forms of collision are 
errors.  They are not the control loop for which we are optimizing.  The 
loop we said we are after is the loop from event detection through 
analysis and response generation to response application.  That loop is 
completely unaffected by caching.

So first, I think folks need to recognize that this discussion is 
revisiting several different and related WG architectural agreements.

Yours,
Joel

---

From: "Russ White" <[email-masked]>
Date: Wed, 4 Nov 2015 19:31:54 -0500

> It is quite clear that caching is not required to meet a <1 second loop.

Given several comments on the list recently that RESTCONF isn't going to be
able to meet these sorts of timers, I'm not certain... Further, I'd like to
see a timer that's tighter than 1sec -- fast reroute needs to happen in
<250ms in many networks. It seems like this particular area leads to "we
don't have enough information right now to decide?" 

> errors.  

Calling an overwrite an error, and stating it isn't something we should
optimize for, are two different things. :-) I don't think they're an error,
even if you consider multiple controllers/clients. I think part of the
problem might be we're thinking about different sorts of data here -- I
don't think having three different processes overwriting the route to
10.1.1.0/24 with a different next hop, and expecting the second highest
priority to take over when the highest priority is removed, is an error --
I'm willing to listen to the case, but I know this sort of thing is
configured all the time in real networks (and not just for static routes). I
can see how it would be considered an error in the case of setting up a new
interface, or modifying a virtual topology, but this is what makes me
suspect we have two different (and heavily interrelated) problems in hand. 

I think we might need to separate the case of "RPC to the RIB" and
"ephemeral overlay state" in some way in our heads, even though we need to
make certain we know how these things interact with one another.

:-)

Russ

---

From: "Joel M. Halpern" <[email-masked]>
Date: Wed, 4 Nov 2015 19:41:00 -0500

If the only problem we want to solve is the RIB, and specifically the 
creation of alternative RIB entries, then a VERY different approach 
would take care of the problem.  The existing RIB models already 
recognize that there are multiple creators.  So if we only need to 
handle alternative RIB entries, that mechanism to could be generalized 
easily to avoid even panes of glass complexity.

But if we want one client to be able to modify part of a RIB entry, or 
if we want to be able to work with things other than RIB entries, then 
all of the issues that were discussed several years ago come back into play.

So if you want to argue that we should solve only the small problem, 
that is defensible.  But it is a big change in our scoping to date.  And 
from where I sit it would not particularly help us solve the larger 
problem, which does need to be addressed.

Yours,
Joel

---

From: "Russ White" <[email-masked]>
Date: Wed, 4 Nov 2015 19:53:28 -0500

> But if we want one client to be able to modify part of a RIB entry, or if
we
> want to be able to work with things other than RIB entries, then all of
the
> issues that were discussed several years ago come back into play.

First -- in terms of granularity -- I don't think anyone should be modifying
parts of a RIB entry. Rather, each "object" should be treated as an "object"
in full. This is something that needs to be addressed in the models -- what
is essential, and what is accidental.

Second -- I'm not arguing one problem should be solved, and others
shouldn't. I'm saying I think we might have two different interrelated
problem sets, both of which need to be solved. For instance, it might be
useful to say something like: "In the RIB, overwrites are not an error; in
all other areas they are." And then define the models and processes to
handle these two different cases. I don't know that using one unified way of
dealing with things is necessarily going to solve all the problems at hand. 

Recognizing this might help us move forward in a coherent way, rather than
stumbling over the same set of problems in different guises each time some
new point is undertaken.

:-)

Russ

---

From: "Susan Hares" <[email-masked]>
Date: Thu, 5 Nov 2015 02:05:16 -0500

Joel: 

<chair hat off> 
I agree that opening up the decision on caching will open a lot of issues.
In my mind, this email thread has shown there may be operational issues with
the "no-caching issue." We are about to send the requirements to the
IESG/NETCONF, and I want to validate the caching/no-caching decision.

Let me give an example. 

Client 1 - priority 5 -- route1 nh 2  time 1
Client 2 - priority 6 -- route1 nh 1  time 2 
Client 3 - priority 4 -- route 1 nh 3 time 3 
Config  -  priority 0 -  route 1 nh 4 time 2

Node = route with subnode nexthop 
If you:
a) save the priority + Nh differences + priority + Time 
b) assign config (lowest priority) + reboot tie 
then the priority resolution can resolve the node. 

I believe this is the basic multiple creators concept you mentioned in your
emails for RIBs. I believe we agree that this is solvable.  I do not see how
this fails to solve a partial RIB. 

Taking the P.I. Topology draft and FB-RIB,  I think a similar sequences can
resolve the problem.  If not, would you let me know the example that does
not fit. 

In our earlier discussions, we stated the grouping of nodes (route, NH,
interfaces) did not fit the model.  In my subsequent discuss with yang
folks, I came to realize a grouping of nodes which are under a particular
point and we could use this. 
   
 Route -----------
     |      \           |
 Prefix  Nh    priority 

If this is true, then the model could provide language that handles the
grouping of variables or things being referred.  This understanding of the
capability (which other I2RS WG members may have known) provides me a way to
model the grouping of parameters.   So, I can perceive a possibility of
solving caching. 

Is this much of my question understandable?  The next part of my question
was to inquire what do you see as some of the issues that we need to
consider on the panes of glass. 
<i2rs individual contributor off > 

Sue 

-----Original Message-----
From: Joel Halpern Direct [mailto:[email-masked]] 
Sent: Wednesday, November 04, 2015 6:42 PM
To: Susan Hares; 'Joel M. Halpern'; 'Andy Bierman'
Cc: [email-masked]; 'Russ White'
Subject: Re: [i2rs] Conversation on Priority and Panes

I do not understand the question in the first part of your email, so I can
not comment on it.

It is quite clear that caching is not required to meet a <1 second loop. 
  In particular, this seems to be a major change in another agreement. 
We as a group concluded that over-write and other forms of collision are
errors.  They are not the control loop for which we are optimizing.  The
loop we said we are after is the loop from event detection through analysis
and response generation to response application.  That loop is completely
unaffected by caching.

So first, I think folks need to recognize that this discussion is revisiting
several different and related WG architectural agreements.

Yours,
Joel

On 11/4/15 5:23 PM, Susan Hares wrote:
> Joel:
>
> On the reading of composite panes, if you believe that ephemeral can 
> be set on any configuration node or as an independent then the 
> following
>
> Config-node-1
> Each ephemeral node could have an ID and a priority.  The composite 
> mode can apply a consistent policy:  (E.g. high priority with tagging for
> I believe this caching is required for the "tight-loop" issue of < 1 
> second response for query/response.
>
> If we define the group issues in the Yang language, then I think we 
> can handle the multiple I2RS ephemeral entries with more memory space. 
> The amount of memory space used by the I2RS caching entries can be set 
> by the implementation in the Agent, and expressed by the model 
> capabilities.  I agree with Andy's original position that Caching will 
> be necessary for high performance on medium to large scale Data models.
>
> Lastly, I am not sure our consensus on the "no-caching" was strong 
> enough to refuse to consider it now.  Meaning less than 10 people in 
> interims or email
> optimize the control loop for cases of improper behavior seems the 
> wrong place to optimize.
>
>
> glass then we are going to need mechanisms to read the composite 
> effect read what I as a client have applied indicate in the response 
> to a write request that the agent has accepted the request, even 
> though it is not actually taking effect.
>
> maintain monitoring of all related activities even though their 
> network application is not in effect.
>
> agent going to store.  So it all probability the client still has to 
> be prepared for being told that not only was its state over-written 
> (which is technically an error) but that it got deleted too.
>
> Yours,
> Joel
>
>> Whether the server rejects lower-priority edits right away, or 
>> whether the agent caches the request in the form of a client-specific 
>> pane, the client still needs to observe the data resources with 
>> pub/sub and decide whether its own particular state is still relevant.
>> IMO the client complexity is the same either way, especially since 
>> the caching is only done if the client requests it.
>>
>>      point of interest, however -- what about backup information? 
>> Some
> vendor
>>      RIBs, for instance, allow a routing process to install not only 
>> the
> best
>>      To convert this to I2RS terms, it does seem worthwhile to me to 
>> have
> the
>>      which is configured locally on the network device. Rather than 
>> trying
> to
>>      at the protocol level. The only thing that needs to be available 
>> is
> the
>>      set, and reduces to an actual protocol mechanism that appears 
>> (?) to
> be
>>      fairly simple, and leaves as much flexibility as possible for 
>> any
> given
>>      agent implementation.
>>
>>      Thoughts?
>>
>>      :-)
>>
>>      Russ
>>

---

From: "Joel M. Halpern" <[email-masked]>
Date: Thu, 5 Nov 2015 02:37:52 -0500

There are two VERY basic questions.

First, is the case of two I2RS clients modifying the same "thing" 
something we consider normal and desirable, or is it an error.  The 
earlier discussions was that it is an error.  In discussing the many 
different kinds of direct and indirect collateral issues that arise, we 
concluded that we could not expect the I2RS agent to be able to 
determine the "right" thing to do in the general case.

I have not seen any evidence that foer the general case we actually know 
this better now.  And I do see a LOT of side effects and implications.

For a pure RIB node solution, we already have the tools for multiple 
writers.  the existing RIB models already have the notion of multiple 
entries whose key differs by who created the entry.  And the notion that 
the RIB manager decides.  We could easily represent each I2RS client as 
a source, and let the RIB manager handle the rest.
But this is a RIB only solution.

My understanding of the WG conclusion was that we wanted a common 
solution for a broad range of cases, including the RIB.  And that we 
were happy to treat collision as an error.  If there is new information 
(I have not seen any) or if the WG has changed its mind, then so be it.

As an example, if one is creating policy entries that make assumptions 
about your route entries being in efect, then having the system change 
what is in effect can produce very strange and unexpected behaviors.

Yours,
Joel

---

From: "Russ White" <[email-masked]>
Date: Thu, 5 Nov 2015 03:10:42 -0500

> something we consider normal and desirable, or is it an error.  The
earlier
> discussions was that it is an error.  In discussing the many different
kinds of
> direct and indirect collateral issues that arise, we concluded that we
could
> not expect the I2RS agent to be able to determine the "right" thing to do
in
> the general case.

So, assume the following --

1. A local BGP process installs a route, 10.1.1.0/24 via 192.168.100.1
2. In order to move traffic off a "hot link" in a fabric, a
client/controller installs a route, 10.1.1.0/24 via 192.168.200.1
3. An attack vector is detected in a flow destined to some host on
10.1.1.0/24 that causes a separate client/controller to install a route,
10.1.1.0/24 via 192.168.150.1 for five seconds

If I were the operator who owned this network, I wouldn't call this an
"error." I would call this "normal operation," -- in fact, the ability to do
the above would be the very reason I would deploy I2RS on the network in the
first place. Further, I would expect the entire process to unwind properly
and _quickly_. I don't care how it happens, I just want the removal of the
second client's route to leave the first client's in the table as the
current route, and the removal of the first client's path leave the BGP
route as the best path. To go farther, why are there client priorities at
all if this is an "error?" If all overwrites of "ephemeral state" are an
error, the agent should simply reject any attempt to overwrite any such
state.

:-)

Russ

---

From: "Susan Hares" <[email-masked]>
Date: Thu, 5 Nov 2015 03:13:09 -0500

Joel: 

Comments below.  Thank you for your comment. 

Sue 

-----Original Message-----
From: Joel M. Halpern [mailto:[email-masked]] 
Sent: Thursday, November 05, 2015 2:38 AM
To: Susan Hares; 'Joel M. Halpern'; 'Andy Bierman'
Cc: [email-masked]; 'Russ White'
Subject: Re: [i2rs] Conversation on Priority and Panes

[agreed] 

First, is the case of two I2RS clients modifying the same "thing" 
something we consider normal and desirable, or is it an error.  The earlier
discussions was that it is an error.  In discussing the many different kinds
of direct and indirect collateral issues that arise, we concluded that we
could not expect the I2RS agent to be able to determine the "right" thing to
do in the general case.
<wg chair hat on> 
[Sue: agreed - this is what we agreed upon.  I am  doing a "final check" on
this concept before I ship off the requirements to the NETCONF.  ]
<wg chair hat off> 

I have not seen any evidence that foer the general case we actually know
this better now.  And I do see a LOT of side effects and implications.
<wg chair hat on> 
[Sue: I am asking for the side effects so that I have an informed list and
implications. You can send these to the list or to me individually.]
<wg chair hat off> 
<individual contributor hat on> 
My work with NETMOD/NETCONF has changed some of my views so I am asking this
question to understand how it aligns with the new things I have learned.  

For a pure RIB node solution, we already have the tools for multiple
writers.  the existing RIB models already have the notion of multiple
entries whose key differs by who created the entry.  And the notion that the
RIB manager decides.  We could easily represent each I2RS client as a
source, and let the RIB manager handle the rest.
But this is a RIB only solution. 
[Sue: In my personal opinion, I think RIB only solution works for Protocol
independent RIB, FB-RIB, and topology.  ]
<individual contributor off>
<wg chair on> 
My understanding of the WG conclusion was that we wanted a common solution
for a broad range of cases, including the RIB.  And that we were happy to
treat collision as an error.  If there is new information (I have not seen
any) or if the WG has changed its mind, then so be it.

We have not changed the WG conclusion to treat collision as an error.  I am
fact finding to see if others still see the list of problems.  I am asking
people to send the problem list to me or the list. 
<WG hat off> 

As an example, if one is creating policy entries that make assumptions about
your route entries being in efect, then having the system change what is in
effect can produce very strange and unexpected behaviors.

<individual contributors hat on> 
Are you concerned about the model changing or the data within a model
changing?  My understanding is the netconf module library yang module can
provide model changing info. 
This is one way that the route policy make assumptions about the route.  If
the data changes regarding policy, these policy nodes can have version.  In
the gated implementation of this concept (10+ years ago), we used a policy
node version in order to solve this problem. 
We also used a binary version of XML. 
<individual contributor hat off> 
<WG chair hat on> 
Joel I appreciate your patience as I discuss pro/cons because I must defend
our joint work to NETCONF. 
<WG chair hat off> 

Yours,
Joel

On 11/5/15 2:05 AM, Susan Hares wrote:
> In my mind, this email thread has shown there may be operational 
> issues with the "no-caching issue." We are about to send the 
> requirements to the IESG/NETCONF, and I want to validate the
caching/no-caching decision.
>
> Let me give an example.
>
> Client 1 - priority 5 -- route1 nh 2  time 1 Client 2 - priority 6 -- 
> route1 nh 1  time 2 Client 3 - priority 4 -- route 1 nh 3 time 3 
> b) assign config (lowest priority) + reboot tie then the priority 
> resolution can resolve the node.
>
> I believe this is the basic multiple creators concept you mentioned in 
> your emails for RIBs. I believe we agree that this is solvable.  I do 
> not see how this fails to solve a partial RIB.
>
> Taking the P.I. Topology draft and FB-RIB,  I think a similar 
> sequences can resolve the problem.  If not, would you let me know the 
> example that does not fit.
>
> folks, I came to realize a grouping of nodes which are under a 
> particular point and we could use this.
>
>   Route -----------
>       |      \           |
>   Prefix  Nh    priority
>
> If this is true, then the model could provide language that handles 
> the grouping of variables or things being referred.  This 
> understanding of the capability (which other I2RS WG members may have
known) provides me a way to
> Is this much of my question understandable?  The next part of my 
> question was to inquire what do you see as some of the issues that we 
> need to consider on the panes of glass.
> We as a group concluded that over-write and other forms of collision 
> are errors.  They are not the control loop for which we are 
> optimizing.  The loop we said we are after is the loop from event 
> detection through analysis and response generation to response 
> application.  That loop is completely unaffected by caching.
>
>> The amount of memory space used by the I2RS caching entries can be 
>> set by the implementation in the Agent, and expressed by the model 
>> capabilities.  I agree with Andy's original position that Caching 
>> will be necessary for high performance on medium to large scale Data
models.
>>> for each client (and unique priorities for each client), because 
>>> some people were talking about sub-milli-sec latency, and I know 
>>> there is no way NETCONF or RESTCONF is going to support this sort of 
>>> tight control
>> loop.
>>>
>>> whether the agent caches the request in the form of a 
>>> client-specific pane, the client still needs to observe the data 
>>> resources with pub/sub and decide whether its own particular state is
still relevant.
>>>       the priority concept within I2RS. The concept does bring up 
>>> one
>> specific
>>>       RIBs, for instance, allow a routing process to install not 
>>> only the
>> best
>>>       some RIB implementations allow the process to place the route in
the
>>>       "backup
>>>       pool." This allows the local RIB process to drop to the "second
best
>>>       path,"
>>>       in terms of priority, so the local RIB doesn't need to query 
>>> the
>> routing
>>>       To convert this to I2RS terms, it does seem worthwhile to me 
>>> to have
>> the
>>>       capability for a local agent to accept an install instruction 
>>> for
> some
>>>       think of this as "panes of glass," though, this would convert it
to
>>>       simply a
>>>       at the protocol level. The only thing that needs to be 
>>> available is
>> the
>>>       2. This ephemeral state was rejected/not installed -- with
potential
>>>       codes
>>>       for why (out of range parameter, etc.) 3. This ephemeral state 
>>> was
> not
>>>       the local agent. It might be that some agents don't know how to
hold
>>>       backup
>>>       information in a backup list. This is fine -- the install can 
>>> just
> be
>>>       rejected without further note. Locally configured information
could
>>>       simply
>>>       considered as always in deciding what to install when any
particular
>>>       action
>>>       is taken.
>>>
>>>       It seems, to me, that this is a simpler way to consider the 
>>> same
>> problem

---

From: Andy Bierman <[email-masked]>
Date: Thu, 5 Nov 2015 09:57:01 -0800

> error, the agent should simply reject any attempt to overwrite any such

I agree that a priority override is not an error. In a multi-client
environment,
client A is not going to know about the other clients added to the system.
This is true whether the clients are primary or secondary behind a broker.

The notification to a client that some or all of its data was just
overridden
is a separate matter from whether the client wants all or part of its
data to be removed or altered.

One proposal to the design team is the notion that the server
supports an advertised (static) number of clients (max client panes).
Each client must be assigned a priority, such that every client
has its own pane, so adding a valid edit does not fail. Activating
the edit is a separate matter. Each client can flush all or part of its own
pane.

:-)

Andy

---

From: "Russ White" <[email-masked]>
Date: Thu, 5 Nov 2015 20:18:22 -0500

> The notification to a client that some or all of its data was just overridden is a
> separate matter from whether the client wants all or part of its data to be
> removed or altered.

I would argue this should be a SHOULD... The idea would be that any client who sets state should be subscribed, but anyone can unsubscribe. This leaves the default to the "right thing," but gives clients the option to sort their own "aggregation" of signaling for efficiency/etc. I'm uncertain about the unsubscribe at this point -- I tend to think we shouldn't bother with "auto-unsubscribe on remove state," as we can't guess either way, and it's just as easy for clients to queue up the removal of state and the unsubscribe one after the other.

Not convinced this is the right answer, though -- thoughts?

> One proposal to the design team is the notion that the server supports an
> advertised (static) number of clients (max client panes).
> Each client must be assigned a priority, such that every client has its own
> pane, so adding a valid edit does not fail. Activating the edit is a separate
> matter. Each client can flush all or part of its own pane.

Just to clarify, I think we're all saying the same thing, just using different terms. If each client must have a unique id, and the client id is directly tied to the priority of the client in terms of installing state, then the pane is also "everything with the same priority," which also means "everything installed by one client." So in relation to the state of a single object, the list of "lower priority" things that aren't installed can be seen as what Joel is calling a "cache." From the perspective of the client, all the state it has installed can be seen as a "pane." 

Does this sound right to everyone?

One point -- it might not be valid for each type of object to have this sort of "backup information." For instance, in the case of a virtual topology built in ephemeral state, it might well be the case (I've not thought it through yet) that it is an error to overwrite lower priority information with higher.

To Joel's point, the backup data (I don't like the term "cache," because it has a different set of connotations for me, but I can deal with it if that's the term everyone settles on) could introduce a lot of complexity unless we also introduce the idea of a set of "things" as a "whole object." In other words, in the case of a route, a "whole object" might include the reachable destination (NLRI in BGP terms), the next hop (which could be a tunnel tail end from the local router's perspective), potentially a stack of markers (like an MPLS label stack), potentially an outbound interface, and potentially a MAC header rewrite. This entire "object" would need to be installed as one "thing," think, to make the "cache" coherent (again, I could be wrong here). To change just the "next hop," a client would need to install "the whole route," I think, or we risk some major issues in coherency. I'm not saying this is possible/not possible today, just throwing out thoughts as they occur to me.

Does all of this make sense?

:-)

Russ

---

From: Joe Clarke <[email-masked]>
Date: Thu, 5 Nov 2015 20:45:36 -0500

> Not convinced this is the right answer, though -- thoughts?

I don't think we should assume that just because the client removes 
state it wants to unsubscribe from the stream.  It may want to watch 
what's going on for events where it can reinsert itself (as an example).

> Does this sound right to everyone?

Yes, this gels with my understanding.

> Does all of this make sense?

It does.  One other point that Jason Coleman, Jason Sterne and I were 
discussing yesterday is how we handle config in this same pane of glass 
model.  That is, if I want to override a route via config, how would 
that work with existing ephemeral state?

The idea proposed was to be able to assign a priority to those [static] 
routes that would, in essence, make them ephemeral with the associated 
priority.  This would allow one from the CLI participate in the I2RS 
process.

There might be other options, but this made a lot of sense at the time.

Joe

---

From: "Russ White" <[email-masked]>
Date: Fri, 6 Nov 2015 22:20:49 -0500

> priority.  This would allow one from the CLI participate in the I2RS

Static routes are always the monster under the bed for this sort of stuff...
The main problem (it seems to me) is how they've been implemented over the
years -- I don't know enough about every implementation to know if treating
them as "just another process" would be painful. Here's where some folks
currently maintaining actual code stepping in and saying, "this would be
really painful," or, "this wouldn't be a problem," would be really helpful.
If it's going to be painful, then we might need some way to indicate which
clients can do what.

I think the main problem with this entire way of looking at things is Joel's
point on complexity, particularly in the agent. It would also be helpful to
try and nail what that complexity looks like, and if there is any way to
reduce it, such as -- 

Would treating a complete set of "things" as a single "object" that cannot
be broken apart be helpful? What do those sets of things look like?

Would limiting this sort of capability to a small number of objects in the
model be helpful in some way? IE, this doesn't seem useful to me for links,
but it does for routes. Can we figure out what the use cases are, and try to
limit the number of objects for which such a capability is to a minimal
number?

Would it be helpful to combine these two concepts? IE, only objects that can
be treated as a "single atomic thing" can have a backup? Is it useful to tie
these concepts together to limit the scope of adding more than one plane to
the object?

I don't know the answers here, just trying to find a middle path that works,
and doesn't introduce too much complexity in return.

:-)

Russ

---

From: Andy Bierman <[email-masked]>
Date: Fri, 6 Nov 2015 20:03:55 -0800

> If it's going to be painful, then we might need some way to indicate which

The RESTCONF server has its own internal mechanisms to deal
with the interface between the protocol and the internal instrumentation.
To some degree, this hides ordering details, etc. from the client.

> be broken apart be helpful? What do those sets of things look like?

This is exactly the sort of thing that the YANG model needs to tell
the server in machine-readable terms.   Implementation of stop-on-error
and continue-on-error is actually harder than it seems.
We never leave invalid data in the running datastore.
Pruning the error data and just accepting the good data is
mostly arbitrary, wrt/ pruning the least amount of the ancestor subtree.
Pruning data can cause a ripple effect wrt/ other validation rules,
such as must/mandatory/min-elements.

Knowing what constitutes a complete set of a "thing" is not in scope in
YANG.
IMO, YANG Packages could solve this problem. ;-)

> limit the number of objects for which such a capability is to a minimal

Yes -- this is the purpose of the "ephemeral true | false" statement
that we want to add to YANG.  It could be added to a data model
(either directly or from another module somehow) to indicate what
an I2RS server is expected to support.

> these concepts together to limit the scope of adding more than one plane to
What if the nature of "single atomic thing" is use-case specific or
even operator preference specific?

As soon as you have 2 use-cases that can share the same data,
the possibility of an edit-collision from normal operation exists.

It also may not really be safe to have the ephemeral client take over
entire sub-trees (e.g. replace instead of merge).  There could be
new nodes or augmentations unknown to the ephemeral client
that may not need to be disabled.

Andy

---

From: Alia Atlas <[email-masked]>
Date: Sun, 8 Nov 2015 09:36:56 -0500

Hi Russ & Andy,

I certainly understand the desire for different behavior when a priority
override happens.
However, this is one area where the working group was extremely clear.  Sue
and I had
ideas of store-if-not-best and handling overwrites and so on.  There was a
very clear
push back against any such complexity.  Feel free to take a read through
the archive.

While it is tempting to expand the scope and functionality of I2RS to
handle this as not
an error, I would ask that we respect the WG consensus and get agreement
and implementations
going on the basics.

We have a serious case of too many saying "This is an interesting soup.
Let's watch it." and
far too few people putting wood on the fire and experimenting.

Regards,
Alia

On Thu, Nov 5, 2015 at 12:57 PM, Andy Bierman <[email-masked]> wrote:

> the edit is a separate matter. Each client can flush all or part of its
> own pane.
>
>
> :-)
>>
>> Russ
>>
>>
>
> Andy
>
>

---

From: "Russ White" <[email-masked]>
Date: Sun, 8 Nov 2015 09:54:13 -0500

> very clear push back against any such complexity.  Feel free to take a read
> through the archive.

IMHO, then, is severely diminished to the point of being non-useful work. If there is no way to store a "backup route," then any use of I2RS to install LFAs, backup tunnels, and even temporary changes in the routing table, are out of bounds, and I2RS becomes "yet another configuration mechanism." 

Russ

---

From: Andy Bierman <[email-masked]>
Date: Sun, 8 Nov 2015 08:51:59 -0800

On Sun, Nov 8, 2015 at 6:36 AM, Alia Atlas <[email-masked]> wrote:

> However, this is one area where the working group was extremely clear.
> Sue and I had
It's hard to get a protocol built here just by throwing requirements over
the wall.
Standards track documents should be past the experimentation stage
(although that is far too common in the IETF). There should be multiple
vendors saying "This is how we solved this problem.  Use our solution."

There are several distinct problems here.

(1) what is an edit overlap?
This is actually a hard problem. Obviously duplicates of running datastore
instances overlap, but not so obvious are the shared ancestor nodes or
data model specific interactions between disjoint objects.  This needs to
be solved
even if there was only 1 client, because running and ephemeral datastores
can overlap.

(2) what needs to happen in the client and server to make the backup data
active?
It concerns me that the implementations of proprietary I2RS use caching
instead of introducing a distributed control loop here.  If you think
caching is
hard, just wait until you try to get 10X - 100X faster performance out of
your
notification system to implement a tight control loop.  There is complexity
in both approaches. The pub/sub work is brand new as well, so it will not
be stable for awhile.

A workaround is to implement a single client, which itself is a broker,
and handles all the caching.  It will try to delete the old data and add
the backup
in 1 step. The alternative will be around 1 sec. latency to install the
backup.

Regards,

Andy

---

From: "Joel M. Halpern" <[email-masked]>
Date: Sun, 8 Nov 2015 12:15:53 -0500

Aince the working group agreement and the request Alia is making is NOt 
to store backup / cache, but to use the control loop to deal with 
changes and errors, I do not follow what your comment 2 is raising?

Yours,
Joel

On 11/8/15 11:51 AM, Andy Bierman wrote:
>
>
> On Sun, Nov 8, 2015 at 6:36 AM, Alia Atlas <[email-masked]
> <mailto:[email-masked]>> wrote:
>
>     Hi Russ & Andy,
>
>     I certainly understand the desire for different behavior when a
>     priority override happens.
>     However, this is one area where the working group was extremely
>     clear.  Sue and I had
>     ideas of store-if-not-best and handling overwrites and so on.  There
>     was a very clear
>     push back against any such complexity.  Feel free to take a read
>     through the archive.
>
>     While it is tempting to expand the scope and functionality of I2RS
>     to handle this as not
>     an error, I would ask that we respect the WG consensus and get
>     agreement and implementations
>     going on the basics.
>
>     We have a serious case of too many saying "This is an interesting
>     soup.  Let's watch it." and
...
> (2) what needs to happen in the client and server to make the backup
> data active?
> hard, just wait until you try to get 10X - 100X faster performance out
> of your
...

---

From: <[email-masked]>
Date: Sun, 8 Nov 2015 12:25:54 -0500

Because the time required to support such a loop is too long to make this work useful in the control plane. We don't have seconds to compute and install an lfa, for instance, or to redirect a ddos stream to a cleaner 

Further, if you think having "planes of configuration," is complex, wait until you try to troubleshoot a muiltistate instability problem with a several second loop time as multiple clients discover, then react to, what other clients are doing. No-one has a "full state set," hence there is no way to either understand what the state should be, or what parts are interacting.

Bottom line -- you can either see the complexity for what it is and manage it, or you can try to throw it over the wall fir someone else to deal with. The second option doesn't really solve the complexity. In fact, it make the problem much worse. Being able to see the potential states in a single place is a crucial piece of the manageability puzzle here.

Russ

Sent from my Windows Phone

-----Original Message-----
From: "Joel M. Halpern" <[email-masked]>
Sent: ‎11/‎8/‎2015 12:15 PM
To: "Andy Bierman" <[email-masked]>; "Alia Atlas" <[email-masked]>
Cc: "Russ White" <[email-masked]>; "[email-masked]" <[email-masked]>; "Susan Hares" <[email-masked]>
Subject: Re: [i2rs] Conversation on Priority and Panes

Yours,
Joel

...
...

---

From: Andy Bierman <[email-masked]>
Date: Sun, 8 Nov 2015 09:41:52 -0800

Hi,

On Sun, Nov 8, 2015 at 9:15 AM, Joel M. Halpern <[email-masked]> wrote:

> Aince the working group agreement and the request Alia is making is NOt to
> store backup / cache, but to use the control loop to deal with changes and
> errors, I do not follow what your comment 2 is raising?
>
>

IMO standards based on operational experience have a better chance of
success.

Andy

---

From: Jeffrey Haas <[email-masked]>
Date: Thu, 19 Nov 2015 15:35:09 -0500

[Catching up on list traffic after being sick for a bit.]

> is essential, and what is accidental.

I think we chatted briefly about this.  While I think we agree in principle
that this is a desired behavior, the methods to implement this are what is
leading to the discordance:

- If we have a yang schema representing a RIB, nothing in netconf/restconf
  prohibits one party with sufficient permissions from tampering with a
  subset of nodes associated with a given RIB entry.

The issue is fundamentally we're allowing different parties to scribble on
top of pieces of configuration.  This is a configuration based mechanism.

- If we only interact with the RIB via something like a RPC call defined in
  yang, then the properties of not interfering with another entry can be
  enforced by the implementation of that RPC.

This is more of an API based mechanism.  The logic implementing the RPC
provides the enforcement.

An API based mechanism addresses the issue, but the question basically comes
down to how to expose the remainder of the associated state for the RIB in
such an implementation.  Do we expose operational state?  If so, we can have
a config=false representation of that state.  

How do we expose the provisioned state?  Do we require the user to issue a
series of RPCs to get the state?  It would be very convenient if it were
present as a config=true set of nodes, but this is the problem we're
avoiding.  What we'd end up with is potentially something similar to what
the OpenConfig group wants: state that is provisioned, but distinct from the
operational state of the system.  In this example, it'd be effectively a
second piece of operational state.

I'm thinking more and more that a significant portion of our problem spaces
come down to some form of the above discussion.  I2RS priority is rather
painful to create and enforce when your interface to provision it is
config=true nodes in an ephemeral datastore.  It's a bit clearer when it is
provisioned via RPC instead.

-- Jeff

---

From: PEDRO ANDRES ARANDA GUTIERREZ <[email-masked]>
Date: Fri, 20 Nov 2015 07:13:04 +0000

HI list,

I’m working on a project were we are also taking on this problem space. Although we are currently OpenFlow based, we are trying to generalise our mechanisms. We hope we may be able to contribute in this discussion.

Best, /PA

PS: answers inline
---
Dr. Pedro A. Aranda Gutiérrez

Technology Exploration -
Network Innovation & Virtualisation
email: pedroa d0t aranda At telefonica d0t com
Telefónica, Investigación y Desarrollo
C/ Zurbarán,12
28010 Madrid, Spain

Fragen sind nicht da, um beantwortet zu werden.
Fragen sind da, um gestellt zu werden.
Georg Kreisler

>a config=false representation of that state.

You either expose the state or you require the different apps to maintain a copy of the state, which is sort of
A) dangerous, because an app could be building a corrupted/biased view of the state if they cannot “spoof" the configuration traffic from other apps and then start creating havoc
B) tedious, because we need all apps to replicate the state

>second piece of operational state.

I see the benefits of that approach. However there is a drawback there and that is the responsiveness and how to control race conditions there:

APP1: gets (and locks) the state, calculates the new state and then writes (and unlocks) the state
APP2: sees the state is locked and waits until it is unlocked to get and lock it to do its operations

That
A) may slow down apps a lot. (Dunno if that can be even desirable to avoid oscillation)
B) rings some bells in the back of my mind regarding deadlocks I studied (long ago) at the university :-) like what happens if APP1 and APP2 access the state at the same time, etc.

>provisioned via RPC instead.

________________________________

Este mensaje y sus adjuntos se dirigen exclusivamente a su destinatario, puede contener información privilegiada o confidencial y es para uso exclusivo de la persona o entidad de destino. Si no es usted. el destinatario indicado, queda notificado de que la lectura, utilización, divulgación y/o copia sin autorización puede estar prohibida en virtud de la legislación vigente. Si ha recibido este mensaje por error, le rogamos que nos lo comunique inmediatamente por esta misma vía y proceda a su destrucción.

The information contained in this transmission is privileged and confidential information intended only for the use of the individual or entity named above. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this transmission in error, do not read it. Please immediately reply to the sender that you have received this communication in error and then delete it.

Esta mensagem e seus anexos se dirigem exclusivamente ao seu destinatário, pode conter informação privilegiada ou confidencial e é para uso exclusivo da pessoa ou entidade de destino. Se não é vossa senhoria o destinatário indicado, fica notificado de que a leitura, utilização, divulgação e/ou cópia sem autorização pode estar proibida em virtude da legislação vigente. Se recebeu esta mensagem por erro, rogamos-lhe que nos o comunique imediatamente por esta mesma via e proceda a sua destruição

---

From: "Russ White" <[email-masked]>
Date: Fri, 20 Nov 2015 07:36:04 -0500

> A) dangerous, because an app could be building a corrupted/biased view of
> the state if they cannot “spoof" the configuration traffic from other apps and
> then start creating havoc

I'm not seeing the planes as being "replicated by various apps," though. If we take a simple model --

Internal device state -- agent -- client(s)

Then the internal device state is the actual running state, the agent is holding the "planes of glass," and each client is holding its desired state, the current state (as the feedback loop), and whatever policies/actions/code it needs to manage. I don't see where client 1 is going to need a full copy of the "planes of glass" the agent is holding, so I don't see any complexity for anyone but the agent here -- and that should be optional.

Consider what happens when some internal device state changes that will (ultimately) trigger some policy on some particular client. There are two options here -- 

1. The agent informs all the clients who have subscribed to that particular event, and then takes the incoming commands as they are processed.
2. The agent has some sort of "cache," or "planes of glass," that allow it to replace information from an internal store as needed, based on predetermined factors it has been informed of by the clients.

I don't see either of these as invalid or "wrong" -- just different responses. In fact, we might want one reaction in some cases, and another in others. There may be some cases where time is of the essence, and hence having the information needed within the agent to make a decision locally is important. There may be other cases where it's not worth the agent locally storing this information; the loop can be a bit longer. We've already admitted this with the inclusion of "backup routes" in the RIB model (which is already an unlimited cache of "backup information" for one particular node in the tree, albeit just for next hops, which is disturbing in many ways from a routing perspective).

The point I think that's being made here is there's no reason not to generalize the "next hop" situation -- the WG has said, "no, we don't want to do that," but the nexthop example has already broken that rule, so I'm not certain that we're even being consistent about our objections, or being clear in what it is we want to do. 

IMHO, there's nothing wrong with the agent being _allowed_ to hold such backup state, or "planes of glass," even if it's for a subset of the entire model, and letting users and vendors work together to decide what makes sense and what doesn't in terms of holding this sort of information locally. I don't see why, as a WG, we should pre-emptively decide that _only_ the nexthop has this sort of speed requirement, and forcing everything else to go through the notify/reply loop. What this is going to lead to, in the long run, I think, is actually more complexity -- as use cases are put forward to support tighter loops for something other than the nexthop, we're going to have to put forward drafts extending the models for each and every case. I don't see how this is simpler than putting the protocol mechanics in place up front to support three responses to an attempt to install some piece of state --

1. Can't do it for reason x
2. Done
3. Stored, but not installed for reason x (same reason codes as above)

And the one additional query -- can you please give me all the state you have stored for this object?

If there's any additional complexity, I'd love to hear about it, because I don't see it right now -- I'm perfectly willing to have my mind changed. But in terms of complexity tradeoff -- and complexity is always a tradeoff -- I see a couple of extra things to define now, versus going through the entire model and somehow building what's already there for nexthop into anything that someone can present a valid use case for.

> APP2: sees the state is locked and waits until it is unlocked to get and lock it

We don't do locks/unlocks -- that's a key point here. This is all atomic. It's not a database, it's more like a web app with a restful interface. 

No locks. Ever. 

:-)

Russ

---

From: PEDRO ANDRES ARANDA GUTIERREZ <[email-masked]>
Date: Fri, 20 Nov 2015 12:55:00 +0000

The danger I see there is that a specific policy applied by one client (app) may trigger unintended changes in a part of the "planes of glass" it is not aware of. Hence a full copy is needed in each client (app) may be needed… This also holds for subscribing to change notifications. How can I subscribe to a change notification of something I’m not or I don’t want to be aware of?

BR,/PA
---

Technology Exploration -

---

From: "Russ White" <[email-masked]>
Date: Fri, 20 Nov 2015 08:11:08 -0500

> The danger I see there is that a specific policy applied by one client (app) may
> trigger unintended changes in a part of the "planes of glass" it is not aware
> of. Hence a full copy is needed in each client (app) may be needed… This also
> holds for subscribing to change notifications. How can I subscribe to a change
> notification of something I’m not or I don’t want to be aware of?

I don't see how that's going to be any different whether the alternate states are held in the agent or on the clients -- either way, doing "x" might trigger some other client to do "y." The only difference is how long it takes for the first client to see the second client doing "y," and then reacting to it by doing "z." Even trying to make certain every client has a full list of all possible conditions there will still be race conditions in the mix, so that won't solve the problem, either. At some point, you're going to hit your head against CAP -- it's just a matter of where, when, and how to manage it. 

The only real solution to this sort of problem is to have a definitive single "metric" that provides an answer to every situation. Just like with a routing protocol, you can't build a distributed system that uses more than one metric without ending up with wedgies (hence BGP). Mike Shand once told me this is an np(complete) problem -- I'm guessing Mike is right on this one. It seems, to me, that this is precisely what the "priority" in this system provides -- in any case, the client with the best priority wins. As no two clients can have the same priority (it's tied to the client id, from what I remember), it "just works."

The one situation you can get in to is when client 1 says "do x," client 2 says, "do y," and you end up with a routing loop or contradictory policies applied. This is going to be a problem no matter where the information is stored. I don't think we can address this at the protocol level -- all the protocol can do is be as accurate about current state as it's installed as possible. The agent should be able to detect some situations like this and refuse to install the offending state. The clients may be able to detect some of this, as well, and fix things. But I don't see how we could specify such a thing in the protocol. Or rather, if we do, when we'd ever be able to stop specifying such things... 

:-)

Russ

---

From: PEDRO ANDRES ARANDA GUTIERREZ <[email-masked]>
Date: Mon, 23 Nov 2015 07:04:26 +0000

Hi,

Re the metric 'problem', just to be more precise in what would be needed, we are looking a metric that grows or decreases monotonically across the whole network. The theoretical grounds for this are in T. Griffin’s and Sobrinho’s work on BGP-4 and that lies already a couple of years ago and that makes the analysis much ‘easier’ (no worries about np(complete) problems, etc.). This could be applied in I2RS at the routing protocol level. So, we could discuss where that sits (should be the clients, right?) and I don’t know if that’s been already done, since I’m quite new to this list.

Now, having “solved” that part of the equation :-) , the part that interests me more is the conflicting clients problem, because this could be generalised to other problem spaces in the SDN area. I do agree that agents should be able to catch offending state before installing it and I’m looking for ways to specify a minimal set of features that need to be supported at protocol level.

Anyone else interested?

BR, /PA
---

Technology Exploration -

---

From: "Russ White" <[email-masked]>
Date: Mon, 23 Nov 2015 07:06:29 -0500

> Re the metric 'problem', just to be more precise in what would be needed,
> we are looking a metric that grows or decreases monotonically across the
> whole network. 

I assume you mean in the routing space, and not in the controller/client space, correct? In terms of a distributed protocol? So, you're saying the delay could use "metrics" between 11 and 20, while the bandwidth could use "metrics" between 21 and 30, etc? And then you just add them all together? That's what IS-IS has done for years... Even BGP really only has one "metric," with following "tie breakers..." So if you have something like weight/local pref/etc, such that they occupy different "spaces" in a bit pattern (something suggested, btw, in the original wide community work, and in other places, as well), you're actually just building a single metric.

You've not "solved" the multiple metric problem -- just done something a little different than EIGRP's K values to combine them into a single metric, which is where you need to be to have the ability to build a single stable SPT/DAG.

> The theoretical grounds for this are in T. Griffin’s and
> Sobrinho’s work on BGP-4 and that lies already a couple of years ago and that
> makes the analysis much ‘easier’ (no worries about np(complete) problems,
> etc.). This could be applied in I2RS at the routing protocol level. So, we could
> discuss where that sits (should be the clients, right?) and I don’t know if
> that’s been already done, since I’m quite new to this list.

This doesn’t apply to the problem at hand, though... You seem to be saying (clarify if wrong) --

1. Give each client some set of bits out of a 128 bit number space (or larger, etc.)
2. Each client can have different "metrics" within their own number space
3. When a client attempts to add some ephemeral state, they use a metric within their "space"
4. The agent can just use the straight number as a relative priority for each potential piece of state

This doesn't do anything different than the current concept of priority between clients, only in the current plan each client only has one priority, rather than multiples. I don't see where this relates to the problem at hand. 
 
> Now, having “solved” that part of the equation :-) , the part that interests me
> more is the conflicting clients problem, because this could be generalised to
> other problem spaces in the SDN area. I do agree that agents should be able
> to catch offending state before installing it and I’m looking for ways to specify
> a minimal set of features that need to be supported at protocol level.
> 
> Anyone else interested?

This is precisely where the problem lies. And this is where you're going to hit the CAP theorem in full force. There are only a few choices --

1. Make the database eventually consistent
2. Shut down accessibility during changes
3. ??

(1) is the idea of either having the agent call back to all the clients when state they installed is overwritten or allowing the agent to locally store some state where it already has the information in hand and install it locally -- the only real difference between these two solutions is the "balance of complexity versus speed..." The entire discussion here is how much additional complexity are we actually adding by doing "panes of glass," which are just locally stored state which isn't currently installed. I'm arguing that there's minimal complexity added that you're not already going to have in the system to allow the agent to store information locally _if_ it chooses to. Jeff is arguing (I think) that the "panes of glass" concept is a coherent way of looking at this problem that will help us understand and manage the complexity in a way that makes sense. Joel is arguing (I think) that this sort of solution is out of the WG charter, and it's too complex. I _think_ I have the three general perspectives right, but I don't know if I really have so... :-)

(2) is the idea of locking the database while you're changing it. This is explicitly outside the scope of I2RS entirely, given we're trying to design something that's atomic/restful. There are a lot of techniques you can use to speed up locking -- row locking, sharding, etc. -- but none of these are interesting from the I2RS perspective. 

:-)

Russ

---

From: PEDRO ANDRES ARANDA GUTIERREZ <[email-masked]>
Date: Tue, 24 Nov 2015 15:32:03 +0000

Hi Russ,

I’m trying to identify the differences between interactions with routing protocols in I2RS and what is purely conflicts between clients. Currently I see too many issues overlapping and I fear that the trees are not letting us see the forest.

So my take on routing protocols and wedgies might have been too compact :-) Let me give it a second try: Stepping outside the I2RS problem space, there is a lot of work that shows that the origin for BGP-4 instability is that our beloved route-maps create metrics that are not monotonically increasing or decreasing and that makes the routing protocols meta-stable. (BTW, I’m the first culprit when it comes to the use of them, I have created more than one wedgie :-P ) Acknowledging that this is a significant (and quite complex) problem for the Internet in general, I feel that it should be treated somewhere else (GROW?).

The perspective I would like to take here is:
- assume that we have two or more clients that produce perfectly sound routing information (no wedgies there)
- assume they talk to the same agent
- now my questions
        1.- is there any way to detect whether the clients produce contradicting/conflicting information?
        2.- is there any way to resolve these contradictions/conflicts?

BR, /PA
---

Technology Exploration -

---

From: "Joel M. Halpern" <[email-masked]>
Date: Tue, 24 Nov 2015 10:36:56 -0500

I believe that determining whether two independent and internally 
consistent sets of changes can, in combination, produce a wedgie or 
other failure is a research problem.
As far as I know, the I2RS working group concluded that it was a 
sufficiently hard problem that we did not expect the I2RS agent to get 
involved in trying to detect, much less remediate, such cross-object 
inconsistencies.

Yours,
Joel

On 11/24/15 10:32 AM, PEDRO ANDRES ARANDA GUTIERREZ wrote:
>> (1) is the idea of either having the agent call back to all the clients when state they installed is overwritten or allowing the agent to locally store some state where it already has the information in hand and install it locally -- the only real difference between these two solutions is the "balance of complexity versus speed..." The entire discussion here is how much additional complexity are we actually adding by doing "panes of glass," which are just locally stored state which isn't currently installed. I'm arguing that there's minimal complexity added that you're not already going to have in the system to allow the agent to store information locally _if_ it chooses to. Jeff is arguing (I think) that the "panes of glass" concept is a coherent way of looking at this problem that will help us understand and manage the complexity in a way that makes sense. Joel is arguing (I think) that this sort of solution is out of the WG charter, and it's too complex. I _think_ I have the th
 r
ee general perspectives right, but I don't know if I really have so... :-)

---

From: Andy Bierman <[email-masked]>
Date: Tue, 24 Nov 2015 08:01:27 -0800

On Tue, Nov 24, 2015 at 7:36 AM, Joel M. Halpern <[email-masked]>
wrote:

> consistent sets of changes can, in combination, produce a wedgie or other
> failure is a research problem.

But the I2RS agent is responsible for identifying an edit collision between
a new low-priority request and existing higher priority edits. That sounds
involved to me.  So the agent cannot possibly solve this problem
yet it is responsible for precisely that in order to implement client
priority.

The agent can see that /foo[42] exists in the request and /foo[42] exists
in the ephemeral datastore, and call that a collision.

However, if /foo[1] or /bar[19] are also part of the "edit", such that
simply replacing
/foo[42] will break things, then the agent will not know. It will simply
overwrite
/foo[42] and leave /foo[1] and /bar[19] in the datastore.

So is the low-priority client responsible for removing /foo[1] and /bar[19]?
Seems the answer is no. The client can go away and there are no cleanup
requirements mentioned in the architecture.

Yours,

Andy

>> I’m trying to identify the differences between interactions with routing
>> protocols in I2RS and what is purely conflicts between clients. Currently I
>> see too many issues overlapping and I fear that the trees are not letting
>> us see the forest.
>>
>> So my take on routing protocols and wedgies might have been too compact
>> :-) Let me give it a second try: Stepping outside the I2RS problem space,
>> there is a lot of work that shows that the origin for BGP-4 instability is
>> that our beloved route-maps create metrics that are not monotonically
>> increasing or decreasing and that makes the routing protocols meta-stable.
>> (BTW, I’m the first culprit when it comes to the use of them, I have
>> created more than one wedgie :-P ) Acknowledging that this is a significant
>> (and quite complex) problem for the Internet in general, I feel that it
>> should be treated somewhere else (GROW?).
>>
>> Para: paag <[email-masked]>, 'Jeffrey Haas' <[email-masked]
>> >
>> CC: "[email-masked]" <[email-masked]>, "'Joel M. Halpern'" <
>> [email-masked]>, 'Andy Bierman' <[email-masked]>, Sue Hares <
>> [email-masked]>
>>> I assume you mean in the routing space, and not in the controller/client
>>> space, correct? In terms of a distributed protocol? So, you're saying the
>>> delay could use "metrics" between 11 and 20, while the bandwidth could use
>>> "metrics" between 21 and 30, etc? And then you just add them all together?
>>> That's what IS-IS has done for years... Even BGP really only has one
>>> "metric," with following "tie breakers..." So if you have something like
>>> weight/local pref/etc, such that they occupy different "spaces" in a bit
>>> pattern (something suggested, btw, in the original wide community work, and
>>> in other places, as well), you're actually just building a single metric.
>>>
>>> You've not "solved" the multiple metric problem -- just done something a
>>> little different than EIGRP's K values to combine them into a single
>>> metric, which is where you need to be to have the ability to build a single
>>> stable SPT/DAG.
>>>
>>> This doesn't do anything different than the current concept of priority
>>> between clients, only in the current plan each client only has one
>>> priority, rather than multiples. I don't see where this relates to the
>>> problem at hand.
>>>
>>> This is precisely where the problem lies. And this is where you're going
>>> to hit the CAP theorem in full force. There are only a few choices --
>>>
>>> (1) is the idea of either having the agent call back to all the clients
>>> when state they installed is overwritten or allowing the agent to locally
>>> store some state where it already has the information in hand and install
>>> it locally -- the only real difference between these two solutions is the
>>> "balance of complexity versus speed..." The entire discussion here is how
>>> much additional complexity are we actually adding by doing "panes of
>>> glass," which are just locally stored state which isn't currently
>>> installed. I'm arguing that there's minimal complexity added that you're
>>> not already going to have in the system to allow the agent to store
>>> information locally _if_ it chooses to. Jeff is arguing (I think) that the
>>> "panes of glass" concept is a coherent way of looking at this problem that
>>> will help us understand and manage the complexity in a way that makes
>>> sense. Joel is arguing (I think) that this sort of solution is out of the
>>> WG charter, and it's too complex. I _think_ I have the th
>>>
>> r
>>> (2) is the idea of locking the database while you're changing it. This
>>> is explicitly outside the scope of I2RS entirely, given we're trying to
>>> design something that's atomic/restful. There are a lot of techniques you
>>> can use to speed up locking -- row locking, sharding, etc. -- but none of
>>> these are interesting from the I2RS perspective.
>>>
>>> :-)
>>>
>>> Russ
>>>
>>>
>> Este mensaje y sus adjuntos se dirigen exclusivamente a su destinatario,
>> puede contener información privilegiada o confidencial y es para uso
>> exclusivo de la persona o entidad de destino. Si no es usted. el
>> destinatario indicado, queda notificado de que la lectura, utilización,
>> divulgación y/o copia sin autorización puede estar prohibida en virtud de
>> la legislación vigente. Si ha recibido este mensaje por error, le rogamos
>> que nos lo comunique inmediatamente por esta misma vía y proceda a su
>> destrucción.
>>
>> The information contained in this transmission is privileged and
>> confidential information intended only for the use of the individual or
>> entity named above. If the reader of this message is not the intended
>> recipient, you are hereby notified that any dissemination, distribution or
>> copying of this communication is strictly prohibited. If you have received
>> this transmission in error, do not read it. Please immediately reply to the
>> sender that you have received this communication in error and then delete
>> it.
>>
>> Esta mensagem e seus anexos se dirigem exclusivamente ao seu
>> destinatário, pode conter informação privilegiada ou confidencial e é para
>> uso exclusivo da pessoa ou entidade de destino. Se não é vossa senhoria o
>> destinatário indicado, fica notificado de que a leitura, utilização,
>> divulgação e/ou cópia sem autorização pode estar proibida em virtude da
>> legislação vigente. Se recebeu esta mensagem por erro, rogamos-lhe que nos
>> o comunique imediatamente por esta mesma via e proceda a sua destruição
>>
>>

---

From: PEDRO ANDRES ARANDA GUTIERREZ <[email-masked]>
Date: Tue, 24 Nov 2015 16:06:04 +0000

Hi Joel,

Generalised conflict detection/resolution is quite a hard problem, I agree on that. But maybe we can start by examining a couple of use cases that we identify.  With the time I have learnt that in many occasions you can start avoiding problems by avoiding the more obvious ones.

Best, /PA
---

Technology Exploration -

---

From: "Joel M. Halpern" <[email-masked]>
Date: Tue, 24 Nov 2015 11:14:48 -0500

The agent is required to detect and report direct collisions.
It is not required or expected to detect indirect collisions, which are 
the complex source of wedgies and other interesting behaviors.

Yours,
Joel

PS: The WG discussed requiring a maintained connection between the 
client and agent, and agreed not to do that.  Which means that no, 
agents do not delete state just because clients have disappeared.

On 11/24/15 11:01 AM, Andy Bierman wrote:
>
>
> On Tue, Nov 24, 2015 at 7:36 AM, Joel M. Halpern <[email-masked]
> <mailto:[email-masked]>> wrote:
>
>     sufficiently hard problem that we did not expect the I2RS agent to
>     get involved in trying to detect, much less remediate, such
>     cross-object inconsistencies.
>
>
>
>         I’m trying to identify the differences between interactions with
>         routing protocols in I2RS and what is purely conflicts between
>         clients. Currently I see too many issues overlapping and I fear
>         that the trees are not letting us see the forest.
>
>         So my take on routing protocols and wedgies might have been too
>         compact :-) Let me give it a second try: Stepping outside the
>         I2RS problem space, there is a lot of work that shows that the
>         origin for BGP-4 instability is that our beloved route-maps
>         create metrics that are not monotonically increasing or
>         decreasing and that makes the routing protocols meta-stable.
>         (BTW, I’m the first culprit when it comes to the use of them, I
>         have created more than one wedgie :-P ) Acknowledging that this
>         is a significant (and quite complex) problem for the Internet in
>         general, I feel that it should be treated somewhere else (GROW?).
>
>         - assume that we have two or more clients that produce perfectly
>         sound routing information (no wedgies there)
>                   1.- is there any way to detect whether the clients
>         produce contradicting/conflicting information?
>         De: Russ White <[email-masked] <mailto:[email-masked]>>
>         Para: paag <[email-masked]
>         <mailto:[email-masked]>>, 'Jeffrey Haas'
>         <[email-masked] <mailto:[email-masked]>>
>         CC: "[email-masked] <mailto:[email-masked]>" <[email-masked]
>         <mailto:[email-masked]>>, "'Joel M. Halpern'"
>         <[email-masked] <mailto:[email-masked]>>, 'Andy
>         Bierman' <[email-masked] <mailto:[email-masked]>>, Sue
>         Hares <[email-masked] <mailto:[email-masked]>>
>             I assume you mean in the routing space, and not in the
>             controller/client space, correct? In terms of a distributed
>             protocol? So, you're saying the delay could use "metrics"
>             between 11 and 20, while the bandwidth could use "metrics"
>             between 21 and 30, etc? And then you just add them all
>             together? That's what IS-IS has done for years... Even BGP
>             really only has one "metric," with following "tie
>             breakers..." So if you have something like weight/local
>             pref/etc, such that they occupy different "spaces" in a bit
>             pattern (something suggested, btw, in the original wide
>             community work, and in other places, as well), you're
>             actually just building a single metric.
>
>             You've not "solved" the multiple metric problem -- just done
>             something a little different than EIGRP's K values to
>             combine them into a single metric, which is where you need
>             to be to have the ability to build a single stable SPT/DAG.
>
>             4. The agent can just use the straight number as a relative
>             priority for each potential piece of state
>
>             This doesn't do anything different than the current concept
>             of priority between clients, only in the current plan each
>             client only has one priority, rather than multiples. I don't
>             see where this relates to the problem at hand.
>
>             This is precisely where the problem lies. And this is where
>             you're going to hit the CAP theorem in full force. There are
>             only a few choices --
>
>             (1) is the idea of either having the agent call back to all
>             the clients when state they installed is overwritten or
>             allowing the agent to locally store some state where it
>             already has the information in hand and install it locally
>             -- the only real difference between these two solutions is
>             the "balance of complexity versus speed..." The entire
>             discussion here is how much additional complexity are we
>             actually adding by doing "panes of glass," which are just
>             locally stored state which isn't currently installed. I'm
>             arguing that there's minimal complexity added that you're
>             not already going to have in the system to allow the agent
>             to store information locally _if_ it chooses to. Jeff is
>             arguing (I think) that the "panes of glass" concept is a
>             coherent way of looking at this problem that will help us
>             understand and manage the complexity in a way that makes
>             sense. Joel is arguing (I think) that this sort of solution
>             is out of the WG charter, and it's too complex. I _think_ I
>             have the th
>
>     r
>             (2) is the idea of locking the database while you're
>             changing it. This is explicitly outside the scope of I2RS
>             entirely, given we're trying to design something that's
>             atomic/restful. There are a lot of techniques you can use to
>             speed up locking -- row locking, sharding, etc. -- but none
>             of these are interesting from the I2RS perspective.
>
>             :-)
>
>             Russ
>
>
>         Este mensaje y sus adjuntos se dirigen exclusivamente a su
>         destinatario, puede contener información privilegiada o
>         confidencial y es para uso exclusivo de la persona o entidad de
>         destino. Si no es usted. el destinatario indicado, queda
>         notificado de que la lectura, utilización, divulgación y/o copia
>         sin autorización puede estar prohibida en virtud de la
>         legislación vigente. Si ha recibido este mensaje por error, le
>         rogamos que nos lo comunique inmediatamente por esta misma vía y
>         proceda a su destrucción.
>
>         confidential information intended only for the use of the
>         individual or entity named above. If the reader of this message
>         is not the intended recipient, you are hereby notified that any
>         dissemination, distribution or copying of this communication is
>         strictly prohibited. If you have received this transmission in
>         error, do not read it. Please immediately reply to the sender
>         that you have received this communication in error and then
>         delete it.
>
>         destinatário, pode conter informação privilegiada ou
>         confidencial e é para uso exclusivo da pessoa ou entidade de
>         destino. Se não é vossa senhoria o destinatário indicado, fica
>         notificado de que a leitura, utilização, divulgação e/ou cópia
>         sem autorização pode estar proibida em virtude da legislação
>         vigente. Se recebeu esta mensagem por erro, rogamos-lhe que nos

---

From: Andy Bierman <[email-masked]>
Date: Tue, 24 Nov 2015 08:28:56 -0800

> the complex source of wedgies and other interesting behaviors.
OK, punting the problem is fair enough.

What happens when the YANG validation for /foo[19] and /bar[1]
now fails because /foo[42] was changed?  Does the agent reject
the edit from the higher priority client?  Does I2RS ignore YANG validation,
assuming the YANG authors really didn't mean what they wrote?

Yours,

Andy

> PS: The WG discussed requiring a maintained connection between the client
> and agent, and agreed not to do that.  Which means that no, agents do not
> delete state just because clients have disappeared.
>
>

---

From: "Joel M. Halpern" <[email-masked]>
Date: Tue, 24 Nov 2015 11:53:32 -0500

That is indeed a good question.

As I understand the WG agreement, some validations of changes are to be 
performed, but as little as possible.  So an agent can reject a change 
for validation, but we hoped to keep it as simple as possible.
I am not sure that hop[e is achievable.

One of the assumptions in making any ephemeral change is that if some 
other changes can affect what a client has done, it is the clients 
responsibility to subscribe to notifications about those other changes, 
and to undertake remedial actions if a problem is introduced.  So if 
/foo[19] and /bar[1] interact with /baz[7] (or /foo[42]) then it is the 
clients job to monitor /baz[7] (or /foo[42]).  The agent can't sort that 
out.
As an example of an ugly case, suppose that /baz[7] has a value created 
by some other I2RS client.  The changes to /foo[19] and /bar[1] might 
depend upon that value.  When the other client removes its changes, the 
value reverts to the config value.  That reversion clearly must take 
place.  That can potential create problems for /foo[19] and /bar[1].  It 
seems to me that we have to leave it to the client to sort that out.

Yours,
Joel

On 11/24/15 11:28 AM, Andy Bierman wrote:
>
>
> On Tue, Nov 24, 2015 at 8:14 AM, Joel M. Halpern <[email-masked]
>     It is not required or expected to detect indirect collisions, which
>     are the complex source of wedgies and other interesting behaviors.
>
>
>         On Tue, Nov 24, 2015 at 7:36 AM, Joel M. Halpern
>         <[email-masked] <mailto:[email-masked]>
>         <mailto:[email-masked] <mailto:[email-masked]>>> wrote:
>
>                  De: Russ White <[email-masked]
>         <mailto:[email-masked]> <mailto:[email-masked]
>         <mailto:[email-masked]>>>
>         <mailto:[email-masked]>
>                  <mailto:[email-masked]
>         <mailto:[email-masked]>>>, 'Jeffrey Haas'
>                  <[email-masked] <mailto:[email-masked]>
>         <mailto:[email-masked] <mailto:[email-masked]>>>
>                  CC: "[email-masked] <mailto:[email-masked]>
>         <mailto:[email-masked] <mailto:[email-masked]>>" <[email-masked]
>         <mailto:[email-masked]>
>                  <mailto:[email-masked] <mailto:[email-masked]>>>, "'Joel
>         M. Halpern'"
>                  <[email-masked] <mailto:[email-masked]>
>         <mailto:[email-masked] <mailto:[email-masked]>>>, 'Andy
>                  Bierman' <[email-masked]
>         <mailto:[email-masked]> <mailto:[email-masked]
>         <mailto:[email-masked]>>>, Sue
>                  Hares <[email-masked] <mailto:[email-masked]>
>         <mailto:[email-masked] <mailto:[email-masked]>>>
>                          Re the metric 'problem', just to be more
>         precise in what
>                          would be needed,
>                          Sobrinho’s work on BGP-4 and that lies already
>         a couple
>                          of years ago and that
>                          discuss where that sits (should be the clients,
>         right?)
>                          and I don’t know if
>                      This doesn’t apply to the problem at hand,
>         though... You
>                      seem to be saying (clarify if wrong) --
>
>                      1. Give each client some set of bits out of a 128
>         bit number
>                      space (or larger, etc.)
>                      2. Each client can have different "metrics" within
>         their own
>                      number space
>                      3. When a client attempts to add some ephemeral
>         state, they
>                      use a metric within their "space"
>                          Now, having “solved” that part of the equation
>         :-) , the
>                          part that interests me
>                          more is the conflicting clients problem,
>         because this
>                          could be generalised to
>                          other problem spaces in the SDN area. I do
>         agree that
>                          agents should be able
>                          to catch offending state before installing it
>         and I’m
>                          looking for ways to specify
>                          a minimal set of features that need to be
>         supported at
>                          protocol level.
>
>                          Anyone else interested?
>
>
>              ee general perspectives right, but I don't know if I really
>         have
>              so... :-)
>
>
>                  o comunique imediatamente por esta mesma via e proceda
>         a sua
>                  destruição
>
>
>

---

From: Andy Bierman <[email-masked]>
Date: Tue, 24 Nov 2015 09:01:37 -0800

On Tue, Nov 24, 2015 at 8:53 AM, Joel M. Halpern <[email-masked]>
wrote:

> performed, but as little as possible.  So an agent can reject a change for
> validation, but we hoped to keep it as simple as possible.
> responsibility to subscribe to notifications about those other changes, and
> to undertake remedial actions if a problem is introduced.  So if /foo[19]
> and /bar[1] interact with /baz[7] (or /foo[42]) then it is the clients job
> to monitor /baz[7] (or /foo[42]).  The agent can't sort that out.
> As an example of an ugly case, suppose that /baz[7] has a value created by
> some other I2RS client.  The changes to /foo[19] and /bar[1] might depend
> upon that value.  When the other client removes its changes, the value
> reverts to the config value.  That reversion clearly must take place.  That
> can potential create problems for /foo[19] and /bar[1].  It seems to me
> that we have to leave it to the client to sort that out.
>
>

This is not much different from clients colliding in persistent config.
Operators can use NACM access control to protect one client or another.
As you have said, these high level collisions are operational errors.
Returning 'access-denied' is better than stomping on data and sorting it
out later.

Andy

>>                          makes the analysis much ‘easier’ (no worries
>> about
>>                          np(complete) problems,
>>                          etc.). This could be applied in I2RS at the
>> routing
>>                          protocol level. So, we could

---

From: "Susan Hares" <[email-masked]>
Date: Mon, 30 Nov 2015 12:49:18 -0500

Pedro: 

I welcome your insights on this problem space. 

Sue 

-----Original Message-----
From: PEDRO ANDRES ARANDA GUTIERREZ [mailto:[email-masked]] 
Sent: Friday, November 20, 2015 2:13 AM
To: Jeffrey Haas; Russ White
Cc: [email-masked]; 'Joel M. Halpern'; 'Andy Bierman'; 'Susan Hares'
Subject: Re: [i2rs] Conversation on Priority and Panes

HI list,

Best, /PA

PS: answers inline
---

Technology Exploration -
Network Innovation & Virtualisation
email: pedroa d0t aranda At telefonica d0t com Telefónica, Investigación y Desarrollo C/ Zurbarán,12
28010 Madrid, Spain

>An API based mechanism addresses the issue, but the question basically 
>comes down to how to expose the remainder of the associated state for 
>the RIB in such an implementation.  Do we expose operational state?  If 
>so, we can have a config=false representation of that state.

>How do we expose the provisioned state?  Do we require the user to 
>issue a series of RPCs to get the state?  It would be very convenient 
>if it were present as a config=true set of nodes, but this is the 
>problem we're avoiding.  What we'd end up with is potentially something 
>similar to what the OpenConfig group wants: state that is provisioned, 
>but distinct from the operational state of the system.  In this 
>example, it'd be effectively a second piece of operational state.

That

>
>I'm thinking more and more that a significant portion of our problem 
>spaces come down to some form of the above discussion.  I2RS priority 
>is rather painful to create and enforce when your interface to 
>provision it is config=true nodes in an ephemeral datastore.  It's a 
>bit clearer when it is provisioned via RPC instead.
>
>-- Jeff
>

---

From: "Susan Hares" <[email-masked]>
Date: Mon, 30 Nov 2015 13:18:39 -0500

Russ: 
[Still Catching up on old email.. ]
The I2RS priority in the architecture requires each client has 1 priority.  The architecture allows for multiple clients to have the same priority (client 1 priority 1; client 2 priority 2).  The resolution for two clients having the same priority is first-one at a priority level wins. 

I agree that the protocol can only be accurate.  The final solution is the metric that resolves all contention on priority and ties on priority.  

Sue 

-----Original Message-----
From: i2rs [mailto:[email-masked]] On Behalf Of Russ White
Sent: Friday, November 20, 2015 8:11 AM
To: 'PEDRO ANDRES ARANDA GUTIERREZ'; 'Jeffrey Haas'
Cc: [email-masked]; 'Joel M. Halpern'; 'Andy Bierman'; 'Susan Hares'
Subject: Re: [i2rs] Conversation on Priority and Panes

> The danger I see there is that a specific policy applied by one client 
> (app) may trigger unintended changes in a part of the "planes of 
> glass" it is not aware of. Hence a full copy is needed in each client 
> (app) may be needed… This also holds for subscribing to change 
> notifications. How can I subscribe to a change notification of something I’m not or I don’t want to be aware of?

:-)

Russ

---

From: "Susan Hares" <[email-masked]>
Date: Mon, 30 Nov 2015 13:20:10 -0500

Russ: 

Andy has mentioned the latency problem of not having back-up routes
repeatedly.   

My message at IETF 94 was encourage this type of thing to get the I2RS
protocol and module out of the "just another configuration model".
Sue 

-----Original Message-----
From: i2rs [mailto:[email-masked]] On Behalf Of Russ White
Sent: Sunday, November 08, 2015 9:54 AM
To: 'Alia Atlas'; 'Andy Bierman'
Cc: [email-masked]; 'Joel M. Halpern'; 'Susan Hares'
Subject: Re: [i2rs] Conversation on Priority and Panes

> was a very clear push back against any such complexity.  Feel free to 
> take a read through the archive.

IMHO, then, is severely diminished to the point of being non-useful work. If
there is no way to store a "backup route," then any use of I2RS to install
LFAs, backup tunnels, and even temporary changes in the routing table, are
out of bounds, and I2RS becomes "yet another configuration mechanism." 

Russ

---

From: "Joel M. Halpern" <[email-masked]>
Date: Mon, 30 Nov 2015 13:37:24 -0500

I think it is important that we either decide that all we care about are 
route entries, or that we still have a broader scope.

If all we want are RIB entries then I suspect we can solve the backup 
problem in a reasonable fashion.
However, when the WG started it was agreed that our scope was 
significantly larger than just the RIB.  With that larger scope, the 
potential for interactions and similar issues make solving the storage 
of backup entries significantly more complex, which is a major part of 
why the WG decided against such an approach.

Yours,
Joel

---

From: "Susan Hares" <[email-masked]>
Date: Mon, 30 Nov 2015 13:49:26 -0500

Pedro: 

<WG hat off> 

For my person insight (as IDR chair) I would like to hear off-list how you think Griffin's work applies to the I2RS context.  Russ has given his reason why.  

I agree I2RS is now discussion the conflict of configuration between clients 

·         assume that we have two or more clients that produce perfectly sound routing information (no wedgies there), 

·          assume they talk to the same agent

For your questions: 

·         I think you must break determine if you are working on the same data model or linked models.  

·         Currently we use the priority metric + First wins. 

I’m interest to hear about alternative ways.

<WG Hat on> 

Please note that I have not changed what will be in I2RS protocol version 1, but it is important to provide additional insight to NETCONF/RESTCONF-I2RS protocol team on what might come  in generation 2 of the I2RS protocol. 

<WG Hat off> 

Sue 

-----Original Message-----
From: PEDRO ANDRES ARANDA GUTIERREZ [mailto:[email-masked]] 
Sent: Tuesday, November 24, 2015 10:32 AM
To: Russ White; 'Jeffrey Haas'
Cc: [email-masked]; 'Joel M. Halpern'; 'Andy Bierman'; 'Susan Hares'
Subject: Re: [i2rs] Conversation on Priority and Panes

Hi Russ,

- now my questions

BR, /PA

---

Technology Exploration -

28010 Madrid, Spain

Georg Kreisler

 

 

-----Mensaje original-----

De: Russ White < <mailto:[email-masked]> [email-masked]>

Para: paag < <mailto:[email-masked]> [email-masked]>, 'Jeffrey Haas' < <mailto:[email-masked]> [email-masked]>

CC: " <mailto:[email-masked]> [email-masked]" < <mailto:[email-masked]> [email-masked]>, "'Joel M. Halpern'" < <mailto:[email-masked]> [email-masked]>, 'Andy Bierman' < <mailto:[email-masked]> [email-masked]>, Sue Hares < <mailto:[email-masked]> [email-masked]>

>> Re the metric 'problem', just to be more precise in what would be 

>> needed, we are looking a metric that grows or decreases monotonically 

>> across the whole network.

>> The theoretical grounds for this are in T. Griffin’s and Sobrinho’s 

>> work on BGP-4 and that lies already a couple of years ago and that 

>> makes the analysis much ‘easier’ (no worries about np(complete) 

>> problems, etc.). This could be applied in I2RS at the routing 

>> protocol level. So, we could discuss where that sits (should be the 

>> clients, right?) and I don’t know if that’s been already done, since I’m quite new to this list.

>This doesn’t apply to the problem at hand, though... You seem to be 

>saying (clarify if wrong) --

>1. Give each client some set of bits out of a 128 bit number space (or 

>larger, etc.) 2. Each client can have different "metrics" within their 

>own number space 3. When a client attempts to add some ephemeral state, they use a metric within their "space"

>4. The agent can just use the straight number as a relative priority 

>for each potential piece of state

>> Now, having “solved” that part of the equation :-) , the part that 

>> interests me more is the conflicting clients problem, because this 

>> could be generalised to other problem spaces in the SDN area. I do 

>> agree that agents should be able to catch offending state before 

>> installing it and I’m looking for ways to specify a minimal set of features that need to be supported at protocol level.

>This is precisely where the problem lies. And this is where you're 

>going to hit the CAP theorem in full force. There are only a few 

>choices --

>1. Make the database eventually consistent 2. Shut down accessibility 

>during changes 3. ??

>when state they installed is overwritten or allowing the agent to 

>locally store some state where it already has the information in hand 

>and install it locally -- the only real difference between these two 

>solutions is the "balance of complexity versus speed..." The entire 

>discussion here is how much additional complexity are we actually 

>adding by doing "panes of glass," which are just locally stored state 

>which isn't currently installed. I'm arguing that there's minimal 

>complexity added that you're not already going to have in the system to 

>allow the agent to store information locally _if_ it chooses to. Jeff 

>is arguing (I think) that the "panes of glass" concept is a coherent 

>way of looking at this problem that will help us understand and manage 

>the complexity in a way that makes sense. Joel is arguing (I think) 

>that this sort of solution is out of the WG charter, and it's too 

>complex. I _think_ I have the three general perspectives right, but I 

>don't know if I really have so... :-)

---

From: "Susan Hares" <[email-masked]>
Date: Mon, 30 Nov 2015 14:03:38 -0500

Joel: 

<WG Hat on>

I2RS agreed the following for the first version of the I2RS protocol and models. 

As far as I know, the I2RS working group concluded that it was a sufficiently hard problem that we did not expect the I2RS agent to get involved in trying to detect, much less remediate, such cross-object inconsistencies.

If I2RS WG members think this is valuable for a future version of the I2RS protocol, it is important to indicate this point.  

If I2RS WG members feel that this mediation is necessarily for a successful I2RS protocol, the chair would like to hear why and in what operational use cases.  Can we deploy simple applications using I2RS without it?  This question should be taken in context of the I2RS requirements process which has complete WG LC, but if you have a major error.  

</WG hat off> 

On whether the statement of:  “whether two independent and internally consistent sets of changes can, in combination, produce a wedgie or other failure is a research problem.” 

I do not think this was Pedro’s point, but I’ll let him respond to this point. 

Sue 

-----Original Message-----
From: Joel M. Halpern [mailto:[email-masked]] 
Sent: Tuesday, November 24, 2015 10:37 AM
To: PEDRO ANDRES ARANDA GUTIERREZ; Russ White; 'Jeffrey Haas'
Cc: [email-masked]; 'Joel M. Halpern'; 'Andy Bierman'; 'Susan Hares'
Subject: Re: [i2rs] Conversation on Priority and Panes

I believe that determining whether two independent and internally consistent sets of changes can, in combination, produce a wedgie or other failure is a research problem.

As far as I know, the I2RS working group concluded that it was a sufficiently hard problem that we did not expect the I2RS agent to get involved in trying to detect, much less remediate, such cross-object inconsistencies.

Yours,

Joel

> - assume that we have two or more clients that produce perfectly sound 

> routing information (no wedgies there)

> email: pedroa d0t aranda At telefonica d0t com Telefónica, 

> Investigación y Desarrollo C/ Zurbarán,12

> Para: paag < <mailto:[email-masked]> [email-masked]>, 'Jeffrey Haas' 

> < <mailto:[email-masked]> [email-masked]>

> CC: " <mailto:[email-masked]> [email-masked]" < <mailto:[email-masked]> [email-masked]>, "'Joel M. Halpern'" 

> < <mailto:[email-masked]> [email-masked]>, 'Andy Bierman' < <mailto:[email-masked]> [email-masked]>, Sue Hares 

> < <mailto:[email-masked]> [email-masked]>

>>> needed, we are looking a metric that grows or decreases 

>>> monotonically across the whole network.

>> 1. Give each client some set of bits out of a 128 bit number space 

>> (or larger, etc.) 2. Each client can have different "metrics" within 

>> their own number space 3. When a client attempts to add some ephemeral state, they use a metric within their "space"

>> (1) is the idea of either having the agent call back to all the 

>> clients when state they installed is overwritten or allowing the 

>> agent to locally store some state where it already has the 

>> information in hand and install it locally -- the only real 

>> difference between these two solutions is the "balance of complexity 

>> versus speed..." The entire discussion here is how much additional 

>> complexity are we actually adding by doing "panes of glass," which 

>> are just locally stored state which isn't currently installed. I'm 

>> arguing that there's minimal complexity added that you're not already 

>> going to have in the system to allow the agent to store information 

>> locally _if_ it chooses to. Jeff is arguing (I think) that the "panes 

>> of glass" concept is a coherent way of looking at this problem that 

>> sense. Joel is arguing (I think) that this sort of solution is out of 

>> the WG charter, and it's too complex. I _think_ I have the th

r

> destinatário, pode conter informação privilegiada ou confidencial e é para uso exclusivo da pessoa ou entidade de destino. Se não é vossa senhoria o destinatário indicado, fica notificado de que a leitura, utilização, divulgação e/ou cópia sem autorização pode estar proibida em virtude da legislação vigente. Se recebeu esta mensagem por erro, rogamos-lhe que nos o comunique imediatamente por esta mesma via e proceda a sua destruição

---

From: "Susan Hares" <[email-masked]>
Date: Mon, 30 Nov 2015 14:14:21 -0500

Pedro: 

[Catching up with last week's email].
<WG chair hat off>  
I am aware of T. Griffin's and Sobrinho's work on a metric that can be applied across the network.  I am interested to see how this could be applied to I2RS protocol.  Could you provide off-list with an example of how this might work. 

On conflicting stages, 
Catching conflicting states on a configuration is single node has been done in many routers for syntax and for context.  However, catching conflicting states across the network is more difficult.  Parts of the configuration may vary from router to router.  For example, if you focus on flow of data packets,  You must have a higher level abstraction that validates the flow of data controlled by routes in routers, interfaces, and policy. 

<WG chair hat on> 
Initial I2RS work examined some of these topics, but we agree this work was out of scope for the version of the protocol.  If you feel there flaw in our first version of I2RS,  please indicate this point.  If you feel this topic should be addressed in a second version, please let me know. 

<WG Chair hat off> 

-----Original Message-----
From: PEDRO ANDRES ARANDA GUTIERREZ [mailto:[email-masked]] 
Sent: Monday, November 23, 2015 2:04 AM
To: Russ White; 'Jeffrey Haas'
Cc: [email-masked]; 'Joel M. Halpern'; 'Andy Bierman'; 'Susan Hares'
Subject: Re: [i2rs] Conversation on Priority and Panes

Hi,

Anyone else interested?

BR, /PA
---

Technology Exploration -

-----Mensaje original-----
De: Russ White <[email-masked]>
Fecha: viernes, 20 de noviembre de 2015, 14:11
Para: paag <[email-masked]>, 'Jeffrey Haas' <[email-masked]>
CC: "[email-masked]" <[email-masked]>, "'Joel M. Halpern'" <[email-masked]>, 'Andy Bierman' <[email-masked]>, Sue Hares <[email-masked]>
Asunto: RE: [i2rs] Conversation on Priority and Panes

>
>> The danger I see there is that a specific policy applied by one 
>> client (app) may trigger unintended changes in a part of the "planes 
>> of glass" it is not aware of. Hence a full copy is needed in each 
>> client (app) may be needed… This also holds for subscribing to change

---

From: "Susan Hares" <[email-masked]>
Date: Mon, 30 Nov 2015 14:23:32 -0500

Joel and Andy: 

For the first version of the I2RS protocol, the WG did agree that the I2RS agent is required to detect and report on collisions.  However, Andy is asking a valid question on how to detect the linkage within models or between models. 

Andy's example is one of the group portions of a model (eg. An I2RS RIB route) or group portions between multiple models (BGP policy + BGP peer or BGP Policy and BGP local route). 

I tried to present this concept at IETF 94 to start this discussion. 

The solutions can be: a)  model structure or language in yang to detect grouping within a model, b) Yang library information to detect modules, or c) something else. 

Sue 

-----Original Message-----
From: i2rs [mailto:[email-masked]] On Behalf Of Joel M. Halpern
Sent: Tuesday, November 24, 2015 11:15 AM
To: Andy Bierman
Cc: Jeffrey Haas; PEDRO ANDRES ARANDA GUTIERREZ; Susan Hares; Russ White; [email-masked]
Subject: Re: [i2rs] Conversation on Priority and Panes

It is not required or expected to detect indirect collisions, which are the complex source of wedgies and other interesting behaviors.

Yours,

Joel

PS: The WG discussed requiring a maintained connection between the client and agent, and agreed not to do that.  Which means that no, agents do not delete state just because clients have disappeared.

On 11/24/15 11:01 AM, Andy Bierman wrote:

> < <mailto:[email-masked]> mailto:[email-masked]>> wrote:

> But the I2RS agent is responsible for identifying an edit collision 

> between a new low-priority request and existing higher priority edits. 

> That sounds involved to me.  So the agent cannot possibly solve this 

> problem yet it is responsible for precisely that in order to implement 

> client priority.

> The agent can see that /foo[42] exists in the request and /foo[42] 

> exists in the ephemeral datastore, and call that a collision.

> simply replacing /foo[42] will break things, then the agent will not 

> know. It will simply overwrite /foo[42] and leave /foo[1] and /bar[19] 

> in the datastore.

> Seems the answer is no. The client can go away and there are no 

> cleanup requirements mentioned in the architecture.

>                   2.- is there any way to resolve these

>         contradictions/conflicts?

>         De: Russ White < <mailto:[email-masked]%20%3cmailto:[email-masked]> [email-masked] <mailto:[email-masked]>>

>         < <mailto:[email-masked]> mailto:[email-masked]>>, 'Jeffrey Haas'

>         < <mailto:[email-masked]%20%3cmailto:[email-masked]> [email-masked] <mailto:[email-masked]>>

>         CC: " <mailto:[email-masked]%20%3cmailto:[email-masked]%3e> [email-masked] <mailto:[email-masked]>" <[email-masked]

>         < <mailto:[email-masked]> mailto:[email-masked]>>, "'Joel M. Halpern'"

>         < <mailto:[email-masked]%20%3cmailto:[email-masked]> [email-masked] <mailto:[email-masked]>>, 'Andy

>         Bierman' < <mailto:[email-masked]%20%3cmailto:[email-masked]> [email-masked] <mailto:[email-masked]>>, Sue

>         Hares < <mailto:[email-masked]%20%3cmailto:[email-masked]> [email-masked] <mailto:[email-masked]>>

>                 Re the metric 'problem', just to be more precise in what

>                 we are looking a metric that grows or decreases

>                 monotonically across the

>                 Sobrinho’s work on BGP-4 and that lies already a couple

>                 makes the analysis much ‘easier’ (no worries about

>                 etc.). This could be applied in I2RS at the routing

>                 discuss where that sits (should be the clients, right?)

>             This doesn’t apply to the problem at hand, though... You

>             1. Give each client some set of bits out of a 128 bit number

>             2. Each client can have different "metrics" within their own

>             3. When a client attempts to add some ephemeral state, they

>                 Now, having “solved” that part of the equation :-) , the

>                 more is the conflicting clients problem, because this

>                 other problem spaces in the SDN area. I do agree that

>                 to catch offending state before installing it and I’m

>                 a minimal set of features that need to be supported at

>     ee general perspectives right, but I don't know if I really have

>         o comunique imediatamente por esta mesma via e proceda a sua

i2rs mailing list

 <mailto:[email-masked]> [email-masked]

 <https://www.ietf.org/mailman/listinfo/i2rs> https://www.ietf.org/mailman/listinfo/i2rs

---

From: "Joel M. Halpern" <[email-masked]>
Date: Mon, 30 Nov 2015 14:30:34 -0500

I am not understanding your comment.
As I understand it, Andy was asking about indirect collisions or side 
effects, where modification of item A by entity B (either an over-write 
or just a modification of something that was in base config) make the 
modification C that was done by entity D incorrect.  We had discussed 
that, and agreed that as a general matter, such indirect problem 
detection was not something we want (wanted) to address.
There are some cases that are represented by YANG constraints.  I don't 
think our decision on this reflects an opinion on what and which YANG 
constraints should be enforced by I2RS.  The issue is generally about 
side effects that are not anticipated by model designers.

The assumption ew made was that at least the application designer had a 
hope of anticipating them, and therefore the client could monitor 
objects when there is a state dependency.  (If no one realizes there is 
a dependence, things will break.  I don't see a way around that.)

Yours,
Joel

On 11/30/15 2:23 PM, Susan Hares wrote:
> Joel and Andy:
>
> For the first version of the I2RS protocol, the WG did agree that the
> I2RS agent is required to detect and report on collisions.  However,
> Andy is asking a valid question on how to detect the linkage within
> models or between models.
>
> Andy's example is one of the group portions of a model (eg. An I2RS RIB
> route) or group portions between multiple models (BGP policy + BGP peer
> or BGP Policy and BGP local route).
>
> The solutions can be: a)  model structure or language in yang to detect
> grouping within a model, b) Yang library information to detect modules,
> or c) something else.
>
> Sue
>
>  >         De: Russ White <[email-masked] <mailto:[email-masked]
> <mailto:[email-masked]%20%3cmailto:[email-masked]>>>
>
>  >         <[email-masked] <mailto:[email-masked]
> <mailto:[email-masked]%20%3cmailto:[email-masked]>>>
>
> <mailto:[email-masked]%20%3cmailto:[email-masked]%3e>" <[email-masked]
>
>  >         <[email-masked] <mailto:[email-masked]
> <mailto:[email-masked]%20%3cmailto:[email-masked]>>>, 'Andy
>
>  >         Bierman' <[email-masked] <mailto:[email-masked]
> <mailto:[email-masked]%20%3cmailto:[email-masked]>>>, Sue
>
>  >         Hares <[email-masked] <mailto:[email-masked]
> <mailto:[email-masked]%20%3cmailto:[email-masked]>>>
>

---

From: "Susan Hares" <[email-masked]>
Date: Mon, 30 Nov 2015 15:10:49 -0500

Joel: 

The "group" concept comes up in I2RS RIB data and FB-RIB entries which use combine entries such as route.  This may be your dependencies.  For I2RS protocol version 1, we must solve this issue. 

Route group:    prefix, next-hop-id, interface 
Highest priority wins: 

If client 1 (priority 4) writes prefix, next-hop-id, and interface 
If client 2 (priority 5) writes next-hop-id 

It impacts the route, and may not be valid if the interface does not lead out to the next-hop. 

Do we simply notify that next-hop is being over-written, or do we notify the route is over-written to client 1?  The argument is that we need to notify client 1 that the components of the route has changed and the next-hop-id is being overwritten.  

The question is:  A) Can we indicate this group in yang?   (by a grouping statement)
B) Should our next-hop change notification and route notification be triggered by this event. 

This is what I mean by group entries.  The application designer will anticipate it, but the data model can also indicate it. 

Sue 

-----Original Message-----
From: i2rs [mailto:[email-masked]] On Behalf Of Joel M. Halpern
Sent: Monday, November 30, 2015 2:31 PM
To: Susan Hares; 'Andy Bierman'
Cc: 'Jeffrey Haas'; 'PEDRO ANDRES ARANDA GUTIERREZ'; 'Russ White'; [email-masked]
Subject: Re: [i2rs] Conversation on Priority and Panes

I am not understanding your comment.
As I understand it, Andy was asking about indirect collisions or side effects, where modification of item A by entity B (either an over-write or just a modification of something that was in base config) make the modification C that was done by entity D incorrect.  We had discussed that, and agreed that as a general matter, such indirect problem detection was not something we want (wanted) to address.
There are some cases that are represented by YANG constraints.  I don't think our decision on this reflects an opinion on what and which YANG constraints should be enforced by I2RS.  The issue is generally about side effects that are not anticipated by model designers.

The assumption ew made was that at least the application designer had a hope of anticipating them, and therefore the client could monitor objects when there is a state dependency.  (If no one realizes there is a dependence, things will break.  I don't see a way around that.)

Yours,
Joel

On 11/30/15 2:23 PM, Susan Hares wrote:
> route) or group portions between multiple models (BGP policy + BGP 
> peer or BGP Policy and BGP local route).
>
> The solutions can be: a)  model structure or language in yang to 
> detect grouping within a model, b) Yang library information to detect 
> modules, or c) something else.
>
> Sue
>
> <[email-masked]

---

From: "Susan Hares" <[email-masked]>
Date: Mon, 30 Nov 2015 15:16:22 -0500

Joel: 

<WG hat off> 
In reference to this example, I was focused on the few examples (routes, filter-based entries, and topology entries) of groups of entries that must be combined.   To have things workable, we must track the priority overwrite with thing such as route notification or next-hop notification.  
 My earlier answer was not about the orthogonal entries that are not grouped). 

Sue 
-----Original Message-----
From: i2rs [mailto:[email-masked]] On Behalf Of Joel M. Halpern
Sent: Tuesday, November 24, 2015 11:54 AM
To: Andy Bierman
Cc: Jeffrey Haas; PEDRO ANDRES ARANDA GUTIERREZ; Susan Hares; Russ White; [email-masked]
Subject: Re: [i2rs] Conversation on Priority and Panes

As I understand the WG agreement, some validations of changes are to be performed, but as little as possible.  So an agent can reject a change for validation, but we hoped to keep it as simple as possible.
I am not sure that hop[e is achievable.

One of the assumptions in making any ephemeral change is that if some other changes can affect what a client has done, it is the clients responsibility to subscribe to notifications about those other changes, and to undertake remedial actions if a problem is introduced.  So if /foo[19] and /bar[1] interact with /baz[7] (or /foo[42]) then it is the clients job to monitor /baz[7] (or /foo[42]).  The agent can't sort that out.
As an example of an ugly case, suppose that /baz[7] has a value created by some other I2RS client.  The changes to /foo[19] and /bar[1] might depend upon that value.  When the other client removes its changes, the value reverts to the config value.  That reversion clearly must take place.  That can potential create problems for /foo[19] and /bar[1].  It seems to me that we have to leave it to the client to sort that out.

Yours,
Joel

On 11/24/15 11:28 AM, Andy Bierman wrote:
> What happens when the YANG validation for /foo[19] and /bar[1] now 
> fails because /foo[42] was changed?  Does the agent reject the edit 
> from the higher priority client?  Does I2RS ignore YANG validation, 
>         cleanup

---

From: Andy Bierman <[email-masked]>
Date: Mon, 30 Nov 2015 12:20:37 -0800

Hi,

I think it is fairly clear that the server will only be able to
detect a direct overlap.  In any case, the client that is getting
data removed needs to use pub/sub to be told what is going on.
The I2RS agent will not set up any pub/sub for any client.
The client must do this itself via  pub/sub configuration.
The agent will just return an error if the edit is not accepted.

The current YANG design is clear: No client is allowed to cause
an edit that will leave the datastore invalid wrt/ YANG validation.
That means that a high priority client WILL LOSE if it attempts
to overwrite partial data in a way that breaks validation.
(This is not what the I2RS arch says to do).  A "solution" might be
to ignore all YANG validation in the ephemeral datastore.
Full YANG validation will be slow and even more complicated
than plain NETCONF. No validation will be a huge security hole.

Andy

On Mon, Nov 30, 2015 at 11:30 AM, Joel M. Halpern <[email-masked]>
wrote:

> effects, where modification of item A by entity B (either an over-write or
> just a modification of something that was in base config) make the
> modification C that was done by entity D incorrect.  We had discussed that,
> and agreed that as a general matter, such indirect problem detection was
> not something we want (wanted) to address.
> constraints should be enforced by I2RS.  The issue is generally about side
> effects that are not anticipated by model designers.
>
> hope of anticipating them, and therefore the client could monitor objects
> when there is a state dependency.  (If no one realizes there is a
> dependence, things will break.  I don't see a way around that.)
>
> Yours,
> Joel
>

---

From: "Susan Hares" <[email-masked]>
Date: Mon, 30 Nov 2015 15:25:39 -0500

Joel, Andy, and Alia. 

Due to a lengthy post-IETF illness, I am catching up on email threads (11/8 to 11/27).   This message is a bit longer to respond to multiple threads. 

<WG chair hat on> 

The specific WG agreement was to focus the first release of the I2RS protocol on control loop with changes and errors.   Our purpose in doing this was to get the I2RS simple protocol with the following: 

·         netconf/restconf with configuration and query/response 

·         secondary feeds (netconf pub/sub, netconf traceability, IPFIX, google protocol buffers),

·         protocol independent models (RIB, FB-RIB, and Topology models, and 

·         initial protocol dependent modules.

The WG’s goal was to enable useful applications to be built with these tools.  

Completing 2015’s work

At IETF 94, I indicated that I2RS 2015-2016 would be wrapping up the initial set of documents(problem, architecture, problem statement, and I2RS protocol requirements) in December.  I2RS will wrap up the initial protocol module drafts (RIB, Topology (RIB, Topology (L2 and L3) and FB-RIB) in December.   Our goal in early 2016 is to get implementations of I2RS agents that support these protocol-independent modules.  

2016 Approved New work 

In 2016, we will also review protocol specific I2RS modules and application drafts. 

2016 Possible Work 

If individuals want to work on proposal to include backup/cache issues in a second I2RS protocol, I am willing to have a small group create such a proposal and initial concept drafts.   Jeff, I and Alia and I would sit down with the individuals working and review their ideas.  After the AD approves, we can bring to the WG as something that should go in the charter.  

After listening to Russ and Andy’s concerns, it is important to capture the operators input, protocol issues (NETCONF, RESTCONF and other protocols), and implementation issues.  I’m interested in getting this discussions started now so that we can provide these early concepts as ideas to the I2RS protocol design team. 

<WG hat off> 

Thanks for all your thoughts and emails this last few weeks.  Each email has helped me gain a larger understanding of the problem. 

Sue 

-----Original Message-----
From: Joel M. Halpern [mailto:[email-masked]] 
Sent: Sunday, November 08, 2015 12:16 PM
To: Andy Bierman; Alia Atlas
Cc: Russ White; [email-masked]; Susan Hares
Subject: Re: [i2rs] Conversation on Priority and Panes

Aince the working group agreement and the request Alia is making is NOt to store backup / cache, but to use the control loop to deal with changes and errors, I do not follow what your comment 2 is raising?

Yours,

Joel

On 11/8/15 11:51 AM, Andy Bierman wrote:

> < <mailto:[email-masked]> mailto:[email-masked]>> wrote:

...

> It concerns me that the implementations of proprietary I2RS use 

> caching instead of introducing a distributed control loop here.  If 

> you think caching is hard, just wait until you try to get 10X - 100X 

> faster performance out of your notification system to implement a 

> tight control loop.  There is complexity in both approaches. The 

> pub/sub work is brand new as well, so it will not be stable for 

> awhile.

...

---

From: "Joel M. Halpern" <[email-masked]>
Date: Mon, 30 Nov 2015 16:26:32 -0500

I agree Andy.  We have our choice of poison.  We can perform full 
validation, which is likely to be slow, but very useful.  Or we can omit 
validation, which introduces risks.  I don't know if I would call those 
security risks in general, although one can probably get some security 
risks out of validation failures.  (We do still assume that permissions 
are checked.)

The conclusion after discussion last time around, and particularly the 
request from the operators who were in the room at the time, was that 
the operators were happy with a situation where they could gain 
significant performance at the risk of shooting themselves in the foot.

Yours,
Joel

On 11/30/15 3:20 PM, Andy Bierman wrote:
> On Mon, Nov 30, 2015 at 11:30 AM, Joel M. Halpern <[email-masked]
>     As I understand it, Andy was asking about indirect collisions or
>     side effects, where modification of item A by entity B (either an
>     over-write or just a modification of something that was in base
>     config) make the modification C that was done by entity D
>     incorrect.  We had discussed that, and agreed that as a general
>     matter, such indirect problem detection was not something we want
>     (wanted) to address.
>     There are some cases that are represented by YANG constraints.  I
>     don't think our decision on this reflects an opinion on what and
>     which YANG constraints should be enforced by I2RS.  The issue is
>     generally about side effects that are not anticipated by model
>     designers.
>
>     The assumption ew made was that at least the application designer
>     had a hope of anticipating them, and therefore the client could
>     monitor objects when there is a state dependency.  (If no one
>     realizes there is a dependence, things will break.  I don't see a
>     way around that.)
>
>     Yours,
>     Joel
>
>         detect
>         <mailto:[email-masked]>] On Behalf Of Joel M. Halpern
>         Cc: Jeffrey Haas; PEDRO ANDRES ARANDA GUTIERREZ; Susan Hares; Russ
>         White; [email-masked] <mailto:[email-masked]>
>           > So is the low-priority client responsible for removing
>         /foo[1] and
>         /bar[19]?
>
>         <mailto:[email-masked]>%20%3cmailto:[email-masked]
>         <mailto:[email-masked]>>>>
>
>         <mailto:[email-masked] <mailto:[email-masked]>
>         <mailto:[email-masked]
>         <mailto:[email-masked]>%20%3cmailto:[email-masked]
>         <mailto:[email-masked]>>>>
>
>         <mailto:[email-masked] <mailto:[email-masked]>>
>         <mailto:[email-masked]
>         <mailto:[email-masked]>%20%3cmailto:[email-masked]
>         <mailto:[email-masked]>%3e>" <[email-masked]
>         <mailto:[email-masked]>
>
>           >         <mailto:[email-masked] <mailto:[email-masked]>>>,
>         "'Joel M. Halpern'"
>
>         <mailto:[email-masked] <mailto:[email-masked]>
>         <mailto:[email-masked]
>         <mailto:[email-masked]>%20%3cmailto:[email-masked]
>         <mailto:[email-masked]>>>>, 'Andy
>
>         <mailto:[email-masked]>%20%3cmailto:[email-masked]
>         <mailto:[email-masked]>>>>, Sue
>
>         <mailto:[email-masked] <mailto:[email-masked]>
>         <mailto:[email-masked]
>         <mailto:[email-masked]>%20%3cmailto:[email-masked]
>         <mailto:[email-masked]>>>>
>
>           >                 that’s been already done, since I’m quite
>         new to this
>         list.
>
>           >
>
>           >
>
>         [email-masked] <mailto:[email-masked]> <mailto:[email-masked]
>         <mailto:[email-masked]>>
>

---

From: PEDRO ANDRES ARANDA GUTIERREZ <[email-masked]>
Date: Tue, 1 Dec 2015 06:11:06 +0000

Catching up myself too…

1) I think I didn’t get my message through WRT Griffin/Sobrinho. I would constrict that kind of effects to the routing protocol (running as an App and sending NETCONF commands to the devices to configure the routes it computes), rather than including this kind of problem in the I2RS space. It seems to me that we would be unnecessarily complicating the problem, (which BTW is complex enough right now)

2)  I see no flaw in v1. However, conflict resolution in general is a topic I’m working on and if work on v2 would include related topics, I’d be glad to try to contribute,

Best, /PA
---

Technology Exploration -