Subject: [netmod] Initial version of the arch draft is available

From: Phil Shafer <[email-masked]>
Date: Thu, 04 Sep 2008 11:48:01 -0400

Please give feedback.

Thanks,
 Phil

------- Forwarded Message

From:     IETF I-D Submission Tool <[email-masked]>
To:       [email-masked]
Subject:  New Version Notification for draft-shafer-netmod-arch-00 
Date:     Thu,  4 Sep 2008 08:49:49 -0700 (PDT)
X-pstn-neptune: 0/0/0.00/0
X-pstn-levels: (S: 7.42583/99.90000 CV:99.9999 R:95.9108 P:95.9108 M:97.0282 C:
     ***98.6951 )
X-pstn-settings: 3 (1.0000:1.0000) s cv gt3 gt2 gt1 r p m c 
X-pstn-addresses: from <[email-masked]> [db-null] 

A new version of I-D, draft-shafer-netmod-arch-00.txt has been successfuly subm
itted by Philip Shafer and posted to the IETF repository.

Filename:	 draft-shafer-netmod-arch
Revision:	 00
Title:		 An Architecture for Network Management
Creation_date:	 2008-09-04
WG ID:		 Independent Submission
Number_of_pages: 23

Abstract:
NETCONF and YANG are pieces of an ambitious plan to improve network
management.  NETCONF gives access to native capabilities of the
devices within a network, defining methods for manipulating
configuration databases, retrieving operational data, and invoking
specific operations.  YANG provides the means to define the content
trafficked via NETCONF, both data and operations.  Using both
technologies, the standards modules can be defined to give
interoperability and commonality to devices, while still allowing
devices to express their unique capabilities.

This document describes how NETCONF and YANG help build network
management applications that meet the needs of network services
providers.  An architecture is described which is friendly to both
devices and applications, to vendors and standards bodies, to young
and to old, to red states and to blue states.  It's a startling
vision, coming to networks near you starting August, 2009.

The IETF Secretariat.

------- End of Forwarded Message

_______________________________________________
netmod mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/netmod

---

From: Andy Bierman <[email-masked]>
Date: Thu, 04 Sep 2008 13:40:55 -0700

> Please give feedback.

I did a first pass, so I probably missed a lot.
The parts about XML namespaces and augment are
what I would expect to see in an NETMOD Arch
document, but most major sections I would expect to
see are not there:

   1) Big picture summary of the NETCONF/NETMOD problem space
      - Evolve from CLI screen-scraping to programmatic device control
      - Take advantage of XML technologies
      - Learn from SMIng and other work already done in the IETF
   2) System Components
      - Manager entity
      - Agent entity
      - NETCONF protocol
      - Transport mappings
      - YANG language
         - configuration data
         - rpc operations
         - notifications
      - YANG to YIN translation
      - YANG to DSDL translation
   3) Sessions
      - User identity
      - Multi-session interactions
   4) Databases
      - Editing
      - Locking
      - Filtered retrieval
      - Validation
   5) Security
   6) Life-cycle Management
   7) Capabilities
   8) Module Conformance

I do not really agree with the architecture behind the picture
on page 11.  If there is any sort of 'transform' between the
standard and the device, it is part of the agent implementation,
not the architecture.  An agent may use all sorts of tools to
get from YANG modules to internal data structures, including
none -- code by hand.

I understand there is going to be a need to migrate from
older proprietary DMs to newer standard DMs -- and that will
always be the case, but it is just an implementation detail.

>  Phil

Andy

---

From: Ladislav Lhotka <[email-masked]>
Date: Fri, 05 Sep 2008 21:35:12 +0200

Hi Phil,

the part about DSDL is amusing but I think it raises a big question
whether it makes sense at all to spend cycles on it. I don't agree with
the view that it is only good for managers and/or people in environments
that haven't been blessed with YANG yet.

My suggestion is to develop the YANG-to-DSDL mapping as a means for
specifying formally the relationship between a YANG model and
corresponding NETCONF PDUs. In the YANG draft, this is only specified in
prose or by examples, so I think this would actually be quite an
important specification.

What do you folks think about it?

Lada

-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C

---

From: Phil Shafer <[email-masked]>
Date: Fri, 05 Sep 2008 15:42:55 -0400

>that haven't been blessed with YANG yet.

I see your point re: the posibility of using DSDL on the device,
but not the second point.  If your tools supported YANG natively,
when would you need the DSDL mapping?

>corresponding NETCONF PDUs.

So the DSDL mapping should cover the PDUs, not the database we are
building?

Thanks,
 Phil

---

From: Phil Shafer <[email-masked]>
Date: Fri, 05 Sep 2008 15:57:56 -0400

>      - Learn from SMIng and other work already done in the IETF

I was told to avoid "how we got here" and "marketing oriented"
discussions.

>      - Transport mappings

So this should cover NETCONF architecture also?  I discuss it
a bit under the "NETCONF" section, but don't get into protocol
operations and transport mappings.

>         - notifications

Will add some words on these.

>      - YANG to DSDL translation

I think this is there.

>      - Multi-session interactions

These are all NETCONF.  Should I be doing a tutorial on NETCONF
features?  I'm not sure "filterer retrieval" is an architectural
issue.

>   6) Life-cycle Management

Life cycle of what?  YANG module, device, app?

>   7) Capabilities

I think these are covered.

>   8) Module Conformance

This should be stressed more.

>get from YANG modules to internal data structures, including

As an application writer, you'll talk to two sorts of boxes.  Those
that support the standard, and those that don't.  If you don't have
some mechanism to transform the standard dialect into a device-specific
dialect, you cannot configure boxes that don't implement the standard.
For emerging standards, this will lose a large percentage of boxes.

Thanks,
 Phil

---

From: Ladislav Lhotka <[email-masked]>
Date: Sat, 06 Sep 2008 09:19:36 +0200

> when would you need the DSDL mapping?

YANG deals with many things, not all of them are relevant for the PDUs
and on the other hand the relationship between a given YANG model and
corresponding PDUs is not formally described in the YANG draft or
elsewhere. So the DSDL draft addressing this relationship would be IMO a
useful complement to the YANG draft.

> So the DSDL mapping should cover the PDUs, not the database we are

Yes. If YANG models are interpreted as contracts, the PDUs are the only
thing that really matters because the NETCONF session is presumably the
only communication channel between the agent and manager. If two
different agents behave in the same way (in terms of PDUs), why should
anyone care about what the agents do under the hood? Another advantage
of the PDU view is that the representation of data is fixed there - it
is XML, and so using XML schema languages for describing the constraints
is a natural choice. In contrast, the representation of an "agent
database" is rather fuzzy - I suspect that for different vendors it may
mean widely different things.

Lada

-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Sat, 6 Sep 2008 10:26:50 +0200

> mean widely different things.

If I write a program, then the program must compile and make sense. It
does not matter how beautiful and complete a single patch is that I am
applying to the program's source code. After applying a patch, I
better check that the program still compiles and makes sense.

The NETCONF edit-config operations essentially applies a patch to a
configuration. While the patch (= PDU) needs to be well formed to be
recognized and applied to the config, what counts after applying the
patch is that the configuration still is valid.

YANG pretty much gets the focus on the configuration datastore right
if you ask me. How a conceptual configuration datastore is implemented
is the implementor's choice.

/js

-- 
Juergen Schoenwaelder           Jacobs University Bremen gGmbH
Phone: +49 421 200 3587         Campus Ring 1, 28759 Bremen, Germany
Fax:   +49 421 200 3103         <http://www.jacobs-university.de/>
_______________________________________________
netmod mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/netmod

---

From: Ladislav Lhotka <[email-masked]>
Date: Sat, 06 Sep 2008 11:27:48 +0200

> patch is that the configuration still is valid.

My idea is to obtain DSDL schemas that specify constraints for the full
(unfiltered) get reply - which is essentially what the DSDL module for
pyang does now, but also for all RPCs and notifications defined in a
YANG model. The assumption is that constraints for other PDUs (filtered
get replies, edit-config) can be derived from these full schemas in a
straightforward way - but this step will be outside the scope of the
DSDL draft, at least for the time being.
> is the implementor's choice.

That's fine, I am not questioning the focus of YANG itself, but if
someone asks "Is this PDU valid under that YANG model?" or "How should
the data from my database appear in get replies?", it's good to be able
to obtain an exact answer, either by inspecting the mapping
specification or by machine validation.

Of course, in the case of PDUs containing partial data (edit-config,
filtered get reply) some constraints may not be verifiable and the
result of applying an edit-config has to be validated internally by the
agent in any case.

Lada

-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Sat, 6 Sep 2008 12:26:05 +0200

> result of applying an edit-config has to be validated internally by the

I expect/hope that filtered gets and edit-configs will be pretty
widely used and perhaps that is why I do personally not see the same
value as you do in partial PDU validation.

/js

---

From: Ladislav Lhotka <[email-masked]>
Date: Sat, 06 Sep 2008 12:59:17 +0200

> value as you do in partial PDU validation.

Neither do I. My point here is that it is necessary to know how the
reply to get without filters may exactly look like for a given YANG
model. Rather than describing it in English and giving a concrete
example of serialised XML as YANG draft does, it is IMO more general and
precise to provide XML schemas. For instance, an example XML document
doesn't tell whether the order of children is fixed or not, whereas the
(RELAX NG) schema does. This should be useful for implementors to make
sure that the PDUs that their agent generates will be properly
understood.

I think it is OK for the YANG draft to describe XML encoding rules using
prose and examples to make it more accessible, but this way it is
difficult to capture all possible cases and combinations, so a formal
schema should be the ultimate norm.

Lada

-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Sat, 6 Sep 2008 15:10:04 +0200

On Sat, Sep 06, 2008 at 12:59:17PM +0200, Ladislav Lhotka wrote:
> Juergen Schoenwaelder p????e v So 06. 09. 2008 v 12:26 +0200:
> > 

I think the YANG mappings to XML should be precise enough to create
interoperable implementations. If there are things and are ambiguous
and where more precision is needed, please tell us so we can fix it.

/js

---

From: Phil Shafer <[email-masked]>
Date: Sun, 07 Sep 2008 22:58:28 -0400

>sure that the PDUs that their agent generates will be properly

We started down this thread asking "If your tools supported YANG
natively, when would you need the DSDL mapping?".  Your answer is
that you need DSDL for validating PDUs, but I don't see this.  If
you have YANG-enabled tools, you can pass them a PDU and say "is
this valid?" and get back an answer.  If don't need DSDL to do this.
A YANG library will understand the XML encoding rules for YANG.

>schema should be the ultimate norm.

If you have to understand DSDL and read the DSDL mapping to understand
how YANG works, we have serious problems in the YANG spec.  The prose
in the YANG spec MUST stand on their own.  If they are weak, we cannot
expect the DSDL mapping document to fix them.

So do we need a formal content spec in the YANG draft?  Do we need
to make an ABNF for YANG content?

   leaf =  "<" leaf-name leaf-attributes ">" leaf-content "</" leaf-name ">"
   leaf-attributes =  leaf-op-attribute
   leaf-op-attribute =  "op" "=" op-attribute-value
   op-attribute-value =  "delete" | "create"
   ... etc ...

Thanks,
 Phil

---

From: Ladislav Lhotka <[email-masked]>
Date: Mon, 08 Sep 2008 09:12:56 +0200

> this valid?" and get back an answer.  If don't need DSDL to do this.

Note that nothing like this is anywhere near horizon, AFAIK. And I think
it would be just waste of cycles to write an XML validator based on
YANG. If we get the YANG->DSDL mapping right, we needn't implement a new
validator for XML (which is far from easy). The XML schema languages
have been designed for this, so why should we reinvent the wheel? People
in this group often stress the need for reusing the know-how of
SNMP/SMI, but isn't it the same case with XML?

> expect the DSDL mapping document to fix them.

I know myself how many times I've had to look to the ABNF for YANG, even
though the statements are supposed to be described fully in the text.
It's the same here: XML schemas provide a concise and formal
specification, apart from the fact that they can be used for machine
validation. A formal spec leaves you no space for hand-waving arguments
whereas prose and simplistic examples do.

Lada

-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C

---

From: Phil Shafer <[email-masked]>
Date: Mon, 08 Sep 2008 08:39:22 -0400

Ladislav Lhotka writes:
>YANG.
 If we get the YANG->DSDL mapping right, we needn't implement a new
>have been designed for this, so why should we reinvent the wheel?

In many (most?) cases, building a configuration engine based on
YANG will involve making a validator.  For my engine, we talk XML
elements while traversing the meta-data (built from our proprietary
YANG equivalent now) so the list of valid element names for any
point in the hierarchy is known, as well as the supported attributes,
etc.  This metadata is needed to build the database, so using an
external XML-based schema is a waste of time.

Using DSDL (or XSD) in an environment where you don't have YANG-based
tools may make sense, but YANG-based tools should handle validation
natively without the need for an external schema language.

More directly:  the YANG spec must give complete details about the
encoding of XML content.  If prose is not sufficient, more direct
mechanisms may be needed, but the YANG spec should not rely on the
DSDL mapping spec to define the XML content mapping for a YANG
module.

>whereas prose and simplistic examples do.

But work on the NETCONF draft showed that no one reads XSD and I
think our experience with relax-ng is even weaker.  A formal spec
in a langauge no one reads is worse than worthless.  The YANG spec
needs to stand on it own, fully describing the configuration database
contents and the contents of the NETCONF PDUs.

Thanks,
 Phil

---

From: Ladislav Lhotka <[email-masked]>
Date: Mon, 08 Sep 2008 15:24:58 +0200

> external XML-based schema is a waste of time.

As a developer, how can you tell that the XML your device produces is
valid? I think it's useful to have a formal way for checking it, either
manually or automatically. Why would be then XML schemas used at all? We
have seen few times already that the "prose" in the YANG spec can be
interpreted in different ways and I think it's an inherent property of
verbal descriptions.

Whether you use the schema directly in a production system or not is a
different story, but IMO it's also possible approach.

> But work on the NETCONF draft showed that no one reads XSD and I

No wonder - that XSD is broken and of no use. It mixes together messages
of the client and server, doesn't take into account capabilities etc.
A schema that can be really used for validating NETCONF PDUs must be
useful for developers almost by definition.

Lada

-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C

---

From: Balazs Lengyel <[email-masked]>
Date: Mon, 08 Sep 2008 16:04:36 +0200

Hello Phil,
Thanks for the draft, nice sometimes even entertaining reading. Some comments:

General:
I 100% agree that the YANG draft MUST be able to define for a given datastore and a given PDU 
if it is valid or not. Naturally validity near always depends on the content of the datastore, 
so as I see it we can not get away from describing the datastore.
OTOH this is one point I would like to see in the DSDL draft: What is it describing. It will be 
interesting to reach agreement on that.

I would emphasize that YANG is both for defining standard modules AND for allowing vendors to 
define proprietary modules. I am missing the latter part.

I would emphasize that YANG tries to be consumable for both humans and applications. We don't 
want something complicated as people wont learn it, but we want something powerful.

I would emphasize that YANG aims to describe more about the devices behavior in a formal way 
compared to earlier stuff e.g. SMI, plain XML or LDAP Schemas.

I would emphasize that we are trying to strike a balance between having one interoperable 
standard and being able to describe the complexities and many options that exist in the real 
world.

I would mention that while officially YANG only defines NETCONF managed data, in practice it is 
and it will be used to define CLI, GUI, documentation aspects, code generation aspects and to 
drive Netconf/YANG toolkits.

IMHO some times the language is a bit too cordial. That is fun too a certain level, ...

Title:
Is this really about Network management or just configuration management or YANG/Netconf based 
management. I don't see much fault or performance management in the draft.

Chapter 3.
I would like to know whether capabilities/supported models/features can change during run-time 
of a node  or just at upgrade/restart ? Adding new stuff, removing it, changing to a new 
version? This must be described either here or in the YANG main draft!!!

Figure:
Didn't you mix up the caption for the two sides?
It is possible that there is no standard view at all anywhere. I would say that will be quite 
common. At least mention it.

Chapter 4.
The parser is just one XML tool that could be mentioned. I would say XSLT is much more 
important e.g. for producing HTML documents, code etc.

Chapter 6.
Is this chapter really needed.

Chapter 7.
I would make this shorter.

Balazs

---

From: Andy Bierman <[email-masked]>
Date: Mon, 08 Sep 2008 07:19:42 -0700

> A YANG library will understand the XML encoding rules for YANG.

Some people seem to think YANG by itself accomplishes
something.  It does not.  Some people think that the ability
to generate a 'pass/fail' validation response is useful.

Since I am interested in using YANG within a NETCONF implementation,
unless the validation tool generates compliant <rpc-error> elements
in every possible NETCONF PDU scenario, it is of no use to me.
Simply generating some blanket "config had errors"
response is not only useless to the manager, it does not comply
with NETCONF.

I also strongly object to the notion that DSDL is needed
to define the YANG to XML mapping for YANG.  That belongs in
the YANG spec, written in text.

Andy