Subject: Re: [netmod] logical systems model

From: Lou Berger <[email-masked]>
Date: Tue, 1 Sep 2015 09:28:22 -0400

On August 28, 2015 5:33:41 PM Andy Bierman <[email-masked]> wrote:

> On Fri, Aug 28, 2015 at 1:31 PM, Lou Berger <[email-masked]> wrote:
>
>> On August 28, 2015 3:53:33 PM Andy Bierman <[email-masked]> wrote:
>>
>> > On Fri, Aug 28, 2015 at 11:31 AM, Alexander Clemm (alex)
<[email-masked]
>> >
>> > wrote:
>> >
>> >>
>> >>
>> >> -----Original Message-----
>> >> From: netmod [mailto:[email-masked]] On Behalf Of Ladislav
>> Lhotka
>> >> Sent: Friday, August 28, 2015 4:31 AM
>> >> To: Martin Bjorklund <[email-masked]>; [email-masked]
>> >> Cc: [email-masked]
>> >> Subject: Re: [netmod] logical systems model
>> >>
>> >>
>> >> .....
>> >> <snip>
>> >>
>> >> NACM, as well as all other modules we have, is based on the assumption
>> of
>> >> a single managed device.
>> >>
>> >> I think it is a typical trend that what once was a single instance
>> becomes
>> >> an array. If we did ietf-routing 20 years ago, there would probably be
>> no
>> >> routing-instance list.
>> >>
>> >> So I think it is a real problem that we can't migrate from a container
>> to
>> >> a list and reuse the container's data model. Groupings might help
>> somewhat
>> >> but they are still not fully reusable, if, for example, they contain
>> >> absolute references.
>> >>
>> >> Lada
>> >> </snip>
>> >>
>> >> One comment, if you forgive the pitch, but this problem / use case
is by
>> >> the way exactly one of the reasons for mounting, which allows you to
>> >> introduce and impose an additional structure on top of an existing
>> >> structure, and "insert" the existing structure into it.  Augmentations
>> etc
>> >> can only add below the hierarchy, there is no way we can put a new
>> >> container or list or whatever on top of an existing structure (without
>> >> replicating the existing structure in a duplicate model), but
peer-mount
>> >> lets you do that.  One use case used in Open Daylight involves
>> organizing/
>> >> inserting device-level information into a network inventory, which is
>> >> basically imposed "on top".
>> >>
>> >>
>> >
>> > I thing YANG Mount would be the best way to solve this problem.
>> > It provides a standard way to do "chroot" and it is flexible.
>> > The mechanics of a "datastore within a datastore" are the
>> > same and independent of the source of the data (local,
>> > remote, virtual, etc.)
>> >
>>
>> So I think an example of how you see this working would helpful.
>>
>> Can one of you give an example of how this word work for a device (which
>> may be physical or virtual) that allocates done resources, say interfaces
>> to one logical entity (router, system, etc) and other resources to a
second
>> entity? And of course I want to manage all with yang and the first and
>> second (sub) entity must be completely independent and ignorant of each
>> other.
>>
>
> Is there an example of this in your draft?
>

Sure, this is what logical-network-element and device view is all about
and the example is actually in the slides from the last ietf that I
pointed to a couple of weeks ago.

> Do you mean an example of what the controller looks like?

No, the server/device.

> Maybe the analogy to chroot is not  universally understood.
>
> The logical system knows only about itself:
>
>     /interfaces
>     /system
>
> The / node is represented by <config> or <data> or <filter> in the
protocol.
>
>    <get-config>
>        <source><running/></source>
>        <filter>
>          <interfaces />
>          <system />
>       </filter>
>    </getconfig>
>
> Each logical system can have its own "eth0" interface or whatever.
> They are mapped to real interfaces in the physical system.
>

It's the control of this mapping that is part of the question. ..

BTW this "physical " system may itself be a virtual device (e.g., VM w/
NFV) . ..

> All operations on the logical system are validated against its own
> virtual datastore.  YANG validation does not work on individual array
> slices -- it only applies to an entire datastore.
>
> On the physical server there needs to be a data model to manage the
> logical servers (as Martin suggested).
>
>  <config>             <--- root on PHY server
>    <interfaces  />   <--------------- contains the real interfaces,
> including eth23
>    <virtual-servers>
>       <virtual-server>
>          <name>vs1</name>
>          <itf-map>
>              <real-itf>eth23</real-itf>
>              <vir-itf>eth0</vir-itf>
>          <itf-map>
>          <more-virtual-server-params ... />
>          <root>                   <----------- YANG mount point (virtual
> server root)
>             <interfaces>
>                <interface>
>                   <name>eth0</name>
>                     ....
>                </interface>
>             </interfaces>
>             <system ... />
>          </root>
>        </virtual-server>
>     </virtual-servers>
>   </config>
>
> On the physical system, the virtual roots are within list entries,
> not at the top of the tree.  The physical server data and/or any
> of the logical servers can be accessed at once.

Okay, so our logical-network-element is basically your virtual-server.root

Think this is an interesting approach that is worth looking at to
address the logical-network-element   (virtual
router/chassis/fabric/<vendor-specific-name>) problem.  I like that it
can go n-levels deep as it is basically recursive.  The challenge will
be dealing with mapping for every top level module that will need it,
e.g., hardware, QoS.  The only major restriction / change from what we
are proposing is that we can allow multiple LNEs to manage one system. 
I'm suspect  loosing this would be a big deal.

I'll discuss with the DT.

> There is no extra indexing cost.  The same data model (e.g
ietf-interfaces)
> can be used in physical and virtual server.  YANG Mount provides a lot
> more than simply identifying /virtual-servers/virtual-server/root
> as the start of an embedded datastore.  As Alex said, not all of it
> needs to be standardized at once.

Sure, but local mounting case does.

Lou

>
> Th "virtual-servers" node is not under each <root> because only
> the physical server loads and supports that module.
>
>
>
> Thanks,
>> Lou
>> >
>>
>
>
> Andy
>
>
>> > Andy
>> >
>> >
>> >
>> > ----------
>> > _______________________________________________
>> > netmod mailing list
>> > [email-masked]
>> > https://www.ietf.org/mailman/listinfo/netmod
>> >
>>

---

From: Lou Berger <[email-masked]>
Date: Tue, 1 Sep 2015 09:45:57 -0400

Alex,

Thanks for the response - see below.

I think the additions of 'mounts' that are themselves configurable (with
support for local and recursion/cascades) is a really interesting and
may turn out to be very useful/important.  Although it does add its own
complexity.

That said, there are some specifics that will need to be addressed to
use this approach: e.g. to quote:
	Mounted data is "read-only" data.
	YANG-Mount does not extend towards RPCs that are defined as
	   part of YANG modules whose contents is being mounted.
	YANG-Mount does not extend towards notifications.
Perhaps most of these limitations can be relaxed for local mounts.

Also handling when a device/server doesn't support local mounts (or is
invalid)

On 8/28/2015 6:16 PM, Alexander Clemm (alex) wrote:
> Hi Lou,
> 
>  
> 
> Andy basically already provided the answer, but just to add to your
> question: yes, there is an example in the draft (appendix A). 
> 

Actually, I was looking for an example on the server/device.  I do like
your example though, as it shows *exactly* the case of a sibling of
/device (i.e, topologies).

Thanks,
Lou

>...

---

From: Martin Bjorklund <[email-masked]>
Date: Wed, 02 Sep 2015 12:42:54 +0200 (CEST)

Andy Bierman <[email-masked]> wrote:
> > to one logical entity (router, system, etc) and other resources to a second

[...]

> The logical system knows only about itself:

This is important.

> The / node is represented by <config> or <data> or <filter> in the protocol.
> 
>    <get-config>

Yes.

>          <root>                   <----------- YANG mount point (virtual

I like this, but I would actually not use mount here.  I don't think
it is necessary.  This would be a model for devices that support
multiple 'virtual-servers' / 'logical-network-elements'.  So in this
model you configure these logical-network-elements and allocate
resources like interfaces etc to them.  For true virtual servers,
you'd also configure the NETCONF server and authentication params,
meaning that each such virtual server has its own config, which is
completely separate from the others.  In this architecture, it would
not be correct to mount all the models in the virtual server list.

/martin

---

From: Lou Berger <[email-masked]>
Date: Wed, 2 Sep 2015 09:17:12 -0400

Martin,

> not be correct to mount all the models in the virtual server list.

We discussed this in the DT and (I think) agreed there is room / need
for both approaches based on device owner/client management model (i.e.,
is the device owner responsible for client config, device owner without
client view.)  Do you have a reference for a model that can be used to
support this, or just thinking one is needed?

Thanks,
Lou

---

From: Andy Bierman <[email-masked]>
Date: Wed, 2 Sep 2015 08:38:33 -0700

On Wed, Sep 2, 2015 at 6:17 AM, Lou Berger <[email-masked]> wrote:

> client view.)

How does this work in YANG exactly?
How does the "foo" container get rooted under "/" in 1 server implementation
and get rooted under "/device" in another server implementation?
I am confused as to why this would be considered a feature and not a bug.

> Do you have a reference for a model that can be used to
Andy

---

From: Paul Borman <[email-masked]>
Date: Wed, 2 Sep 2015 08:59:14 -0700

It depends on if you want to be able to just use YANG for schemas, or if
you want to force people to use another mechanism on top of YANG to make it
flexible enough for complete schemas.

For example, an interfaces schema may end up with paths like (I am not
using full prefixed YANG paths, so please don't be confused by that):

/interfaces/eth0/...

But, if you want to report everything about a device, you would probably
want:

/device/interfaces/eth0/..
/device/protocols/tcp/...

Now step back, you actually want to contain everything about all your
devices:

/router1.me.com/interfaces/eth0/...
/router2.me.com/interfaces/eth1/...

And now you realize you have other things besides network devices to add to
your schema:

/network/router1.me.com/interfaces/eth0/...
/server/server1.me.com/memory/...

And then you want to group by continent and datacenter:

/na/dc/network/router1.me.com/interfaces/eth0/...
/na/dc/server/server1.me.com/memory/...

And since we live in a global community:

/me.com/na/dc/network/router1.me.com/interfaces/eth0/...
/me.com/na/dc/server/server1.me.com/memory/...

These are just examples to show the point, and the point is simple, if you
try to authoritatively say where the root of the tree must be, either a)
your schema cannot be incorporated, or b) you have to build something on
top.

Why should an organization expend the incredible amount of effort it takes
to describe things in YANG just to have the results limited?  I honestly
don't care about NETCONF, I care about a schema language we can use, our
network device vendors can support, and is flexible enough that we can
organize our tree how it makes sense without forcing our global structure
onto our vendors.  YANG is coming up short for this goal.

Thanks,

    -Paul

---

From: Lou Berger <[email-masked]>
Date: Wed, 2 Sep 2015 12:11:59 -0400

> How does this work in YANG exactly?
 I can see how yang might be used to manage an NFV 'hypervisor' that
would spin up some base level of management capability, but would like
to hear what Martin was thinking as he's the one that proposed this...

Lou

---

From: Martin Bjorklund <[email-masked]>
Date: Wed, 02 Sep 2015 21:03:13 +0200 (CEST)

> support this, or just thinking one is needed?

I don't know if it is possible to find agreement among hypervisor
implementations etc to find a common model.  But from a YANG language
perspective I think it is pretty straightforward.  Since each virtual
server is truly virtual, there is no need for the mount functionality
in such a model.

As for the other model with logical systems within a real system,
where each logical system tries to present its own 'system view' of
the config and state, even though in reality there is just one config,
I think a mechanism like 'mount' would be useful.  If it was done with
'mount' instead of the proposed model in
draft-rtgyangdt-rtgwg-device-model-00, it doesn't cost anything for
the 99% (more?) of all systems that do not have this kind of logical
systems, and data models would not have to augment the
/device/logical-network-elements/logical-network-element path.

/martin

---

From: Martin Bjorklund <[email-masked]>
Date: Wed, 02 Sep 2015 21:06:24 +0200 (CEST)

> your schema cannot be incorporated, or b) you have to build something on

I agree.  This illustrates that we should not try to enforce all
models to be under /device.  If the interface model is under
/interfaces, it can then be mounted in different structural models,
serving different purposes, as shown above.

/martin

---

From: Lou Berger <[email-masked]>
Date: Wed, 2 Sep 2015 16:17:02 -0400

Martin,

On 9/2/2015 3:03 PM, Martin Bjorklund wrote:
>  If it was done with

This is a compelling point (at least to me).  Of course there's lots of
details to flush out to make this work, but seems like it's worth
exploring this approach in more detail. 

The initial set of issues I see were in an earlier e-mail:
 
On 9/1/2015 9:45 AM, Lou Berger wrote:
> That said, there are some specifics that will need to be addressed to > use this approach: e.g. to quote: > Mounted data is "read-only" data.
> YANG-Mount does not extend towards RPCs that are defined as > part of
YANG modules whose contents is being mounted. > YANG-Mount does not
extend towards notifications. > Perhaps most of these limitations can be
relaxed for local mounts. > > Also handling when a device/server doesn't
support local mounts (or is > invalid) Lou

---

From: Martin Bjorklund <[email-masked]>
Date: Thu, 03 Sep 2015 08:46:21 +0200 (CEST)

> support local mounts (or is > invalid) Lou

If this is viewed as a way to incorporate the schema defined in
another module, rather than tie it to run-time behavior, there is no
need to make it read-only.

Further, with the addition of 'action' and inline notifications in
YANG 1.1, we can make this handle RPCs and notifications at the
top-level as well.  For example:

  module system {
    ...
    rpc shutdown { ... }
  }

  module controller {
    ...
    list device {
      key name;
      leaf name { ... }
      leaf addres { ... }

      mount system;
    }
  }

would behave similar (forgetting namespaces etc) to as if it was
defined as:

  module controller {
    ...
    list device {
      key name;
      leaf name { ... }
      leaf addres { ... }

      action shutdown { ... }
    }
  }

Maybe we should try to find another term than 'mount',
since it seems people think about the specific solution in
draft-clemm-netmod-mount.  It might also be necessary to 'mount' parts
of a module.

/martin

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Mon, 7 Sep 2015 13:40:29 +0200

> /router2.me.com/interfaces/eth1/...

Instance identification does not really work by putting 'names' into
paths. Neither in NETCONF nor in RESTCONF. If your starting point is
based on wrong assumptions, then the conclusions may be wrong as well.

/js

-- 
Juergen Schoenwaelder           Jacobs University Bremen gGmbH
Phone: +49 421 200 3587         Campus Ring 1 | 28759 Bremen | Germany
Fax:   +49 421 200 3103         <http://www.jacobs-university.de/>

---

From: Ladislav Lhotka <[email-masked]>
Date: Mon, 7 Sep 2015 16:37:34 +0200

> On 03 Sep 2015, at 08:46, Martin Bjorklund <[email-masked]> wrote:
> 

Right, this discussion is about an alternative method for assembling a schema from modular parts whereas the mount draft deals with mediating access to remote datastores, which is something very different.

Lada

--
Ladislav Lhotka, CZ.NIC Labs
PGP Key ID: E74E8C0C

---

From: Paul Borman <[email-masked]>
Date: Tue, 8 Sep 2015 08:40:52 -0700

> based on wrong assumptions, then the conclusions may be wrong as well.

XPATH has a mechanism for putting them in, but for demonstration the layout
of the tree it is much clearer to use the very standard idiom of normal
path names.  I believe I even pointed out that my path names were not in
the correct form, but that is irrelevant to the issue.