Subject: [Netconf] I-D Action:draft-ietf-netconf-partial-lock-08.txt

From: [email-masked]
Date: Wed, 3 Jun 2009 01:45:01 -0700 (PDT)

A New Internet-Draft is available from the on-line Internet-Drafts directories.
This draft is a work item of the Network Configuration Working Group of the IETF.

	Title           : Partial Lock RPC for NETCONF
	Author(s)       : B. Lengyel, M. Bjorklund
	Filename        : draft-ietf-netconf-partial-lock-08.txt
	Pages           : 31
	Date            : 2009-06-03

The NETCONF protocol defines the lock and unlock RPCs, used to lock
entire configuration datastores.  In some situations, a way to lock
only parts of a configuration datastore is required.  This document
defines a capability-based extension to the NETCONF protocol for
locking portions of a configuration datastore.

A URL for this Internet-Draft is:
http://www.ietf.org/internet-drafts/draft-ietf-netconf-partial-lock-08.txt

Internet-Drafts are also available by anonymous FTP at:
ftp://ftp.ietf.org/internet-drafts/

Below is the data which will enable a MIME compliant mail reader
implementation to automatically retrieve the ASCII version of the
Internet-Draft.

---

From: "Bert Wijnen \(IETF\)" <[email-masked]>
Date: Wed, 3 Jun 2009 13:28:34 +0200

WG particpants,

Unfortunately, not too many people have given a clear statement of their 
opinion on the question if they could accept Balazs' reasoning w.r.t. the
partial lock issues (see below).

But, based on the comments received, We (WG chairs) have asked
Balazs to do a new rev that explains as much as possible what can 
and cannot be done and what impacts it has?

Balazs has now done so and revision draft-ietf-netconf-partial-lock-08.txt
is now available. Revision 7 was already handed to our AD for review
and IETF Last Call, and we believe that this new revision addresses
(or at least explains) the last issues that came up before our AD could 
do an IETF LC for rev 07.

We have asked Dan (our AD) to pick up revision 08 for IETF Last Call.

I assume all WG members are now OK with this revision. We have
(in the WG chairs views) consensus (at least rough) consensus.
If anyone still has concerns, you can always post them to our NETCONF
mailing list and.or raise them during IETF Last Call.

Thanks you all.
Bert and Mehmet

----- Original Message ----- 
From: Bert Wijnen (IETF) 
To: Balazs Lengyel ; netconf mailing list 
Sent: Tuesday, May 19, 2009 8:51 PM
Subject: Re: [Netconf] Partial-lock issues

Thanks Balazs for summarizing.

I believe I (as a technical contributer) can accept your reasoning.

Speaking as document shepherd and WG co-chair I would like to
urge all WG members to state their opinion (or comments) as well.

Bert
  ----- Original Message ----- 
  From: Balazs Lengyel 
  To: netconf mailing list 
  Sent: Tuesday, May 19, 2009 12:01 PM
  Subject: [Netconf] Partial-lock issues

  Hello,
  Some people brought it up that partial-locking without partial commit is not a good solution as
  it can lead to dead-locks. For one last time I try to list the reasons why we included candidate.

  Please state if you can accept the reasoning below or whether you think we should allow
  partial-locking ONLY for the running configuration?

  1) As Washam pointed out just by having access control and without using any locks we face a
  dead-lock problems:
  - A only has access to /foo, and edits it in candidate
  - B only has access to /bar, and edits it in candidate
  - A terminates it's session
  - Now B can not issue <discard-changes> because it can not modify /foo in the candidate
  - B can not issue <commit> because it can not modify /foo in the running
  If A and/or B uses partial-locking that will NOT make this situation WORSE, as the lock is
  automatically released at the end of the session. The real problem is caused by access control.

  2) When partial-locking is used correctly, there is no danger of dead-lock, and locking does
  help to protect the individual operator's activities. (In the next sequence we presume access
  control will not block any of the actions.)

  - Operator A locks everything it needs both in candidate and running in one operation e.g. /foo
  - A edits the locked part in the candidate
  - A issues commit, if this fails due to lock conflicts, it simply releases the lock, and let's
  any concurrent operator (e.g. B) work on candidate; B will later commit changes done by A as
  well. Lock conflicts can include operator B locking /bar in the running configuration
  - Operator B works the same way on another area e.g. /bar
  - The operator that finishes editing last will successfully commit all changes to running (by
  this time there are no other locks)

  3) Later we might introduce partial commit, in which case partial-locking would be very much
  needed/useful

  4) Partial-locking does no evil

  5) It is nice to have the feature available on all three datastores

  So we propose to keep partial-locking on candidate, but as this is only secondary use-case if
  the workgroup decides it can be removed.

  This question has already been raised a number of times, so we would like to establish the
  workgroup
  consensus and follow it. PLEASE INDICATE WHETHER PARTIAL-LOCKING ON CANDIDATE SHOULD STAY OR
  SHOULD BE REMOVED !!!!

  Balazs

---

From: "David B Harrington" <[email-masked]>
Date: Wed, 3 Jun 2009 12:54:14 -0400

Hi,
 
I think there is a real problem that exists, and Balaz's reasoning
effectively says "let's just ignore the conflict with access control;
then we can make believe the problem does not exist." This does not
make the problem go away.
 
The problem is that the scope of the lock and the scope of the commit
are different.
I believe that for partial lock on candidiate to work, you need a
corresponding partial commit.
This could be handled using an explicit command for partial commit.
It could be handled by allowing multiple candidiates, each containing
a partial config that is specified as a parameter to the partial lock
(or to a partial-copy-config), with what is effectively a global lock
of the partial config and a global commit of that partial config.
The lock and the commit need to match scope. If they do not match, you
will have problems.
 
If the WG chooses to not have a partial commit that corresponds to the
partial lock, or multiple candidiate sandboxes that can allow an
operator to independently lock, edit, and commit a partial config,
then I think the partial lock for candidate should be removed. 
 
That is not my preferred solution, because I believe operators will
need to lock and modify and commit partial configs, but removing the
partial lock from candidiate is probably the easiest for now.
 
dbh

  _____  

From: [email-masked] [mailto:[email-masked]] On
Behalf Of Bert Wijnen (IETF)
Sent: Wednesday, June 03, 2009 7:29 AM
To: [email-masked]
Cc: Ron Bonica
Subject: [Netconf] New draft-ietf-netconf-partial-lock-08.txt goes to
AD andIETF Last Call

WG particpants,
 
Unfortunately, not too many people have given a clear statement of
their 
opinion on the question if they could accept Balazs' reasoning w.r.t.
the
partial lock issues (see below).

Balazs has now done so and revision
draft-ietf-netconf-partial-lock-08.txt
is now available. Revision 7 was already handed to our AD for review
and IETF Last Call, and we believe that this new revision addresses
(or at least explains) the last issues that came up before our AD
could 
do an IETF LC for rev 07.

 
Thanks you all.
Bert and Mehmet

----- Original Message ----- 
From: Bert Wijnen (IETF) 
To: Balazs Lengyel ; netconf mailing list 
Sent: Tuesday, May 19, 2009 8:51 PM
Subject: Re: [Netconf] Partial-lock issues

 
 
Bert

----- Original Message ----- 
From: Balazs Lengyel 
To: netconf mailing list 
Sent: Tuesday, May 19, 2009 12:01 PM
Subject: [Netconf] Partial-lock issues

Hello,
Some people brought it up that partial-locking without partial commit
is not a good solution as
it can lead to dead-locks. For one last time I try to list the reasons
why we included candidate.

Please state if you can accept the reasoning below or whether you
think we should allow
partial-locking ONLY for the running configuration?

1) As Washam pointed out just by having access control and without
using any locks we face a
dead-lock problems:
- A only has access to /foo, and edits it in candidate
- B only has access to /bar, and edits it in candidate
- A terminates it's session
- Now B can not issue <discard-changes> because it can not modify /foo
in the candidate
- B can not issue <commit> because it can not modify /foo in the
running
If A and/or B uses partial-locking that will NOT make this situation
WORSE, as the lock is
automatically released at the end of the session. The real problem is
caused by access control.

2) When partial-locking is used correctly, there is no danger of
dead-lock, and locking does
help to protect the individual operator's activities. (In the next
sequence we presume access
control will not block any of the actions.)

- Operator A locks everything it needs both in candidate and running
in one operation e.g. /foo
- A edits the locked part in the candidate
- A issues commit, if this fails due to lock conflicts, it simply
releases the lock, and let's
any concurrent operator (e.g. B) work on candidate; B will later
commit changes done by A as
well. Lock conflicts can include operator B locking /bar in the
running configuration
- Operator B works the same way on another area e.g. /bar
- The operator that finishes editing last will successfully commit all
changes to running (by
this time there are no other locks)

3) Later we might introduce partial commit, in which case
partial-locking would be very much
needed/useful

So we propose to keep partial-locking on candidate, but as this is
only secondary use-case if
the workgroup decides it can be removed.

This question has already been raised a number of times, so we would
like to establish the
workgroup
consensus and follow it. PLEASE INDICATE WHETHER PARTIAL-LOCKING ON
CANDIDATE SHOULD STAY OR
SHOULD BE REMOVED !!!!

Balazs

---

From: Andy Bierman <[email-masked]>
Date: Wed, 03 Jun 2009 10:09:51 -0700

David B Harrington wrote:
> then we can make believe the problem does not exist." This does not make 
> the problem go away.
>  
> It could be handled by allowing multiple candidiates, each containing a 
> partial config that is specified as a parameter to the partial lock (or 
> to a partial-copy-config), with what is effectively a global lock of the 
> partial config and a global commit of that partial config.

As I have stated before, I strongly object to changing
the NETCONF database architecture without a complete
re-charter and development of a NETCONF Architecture
document first -- one that does not ignore access control.

NETCONF databases and SQL databases do different things,
and are intended for different target environments.
For NETCONF servers, CM database manipulation is a 2nd or 3rd
order problem, as opposed to the *only* problem for
an SQL server.

Partial commit does not work for <candidate>.
It can have the same effect as not using any lock at all,
which IMO fails claim #4 below (do no evil).

I'm not so sure <partial-lock> works for <running> either.
What if :startup is supported and a <copy-config> is
needed to actually save the changes?  That is a global
operation, which means the independent writers use-case
is a myth.  They may have concurrent edit capability
for <running>, but the copy to <startup> still needs
to be coordinated.

> That is not my preferred solution, because I believe operators will need 
> to lock and modify and commit partial configs, but removing the partial 
> lock from candidiate is probably the easiest for now.
>  
> dbh
>  

Andy

>  
> 
>     ------------------------------------------------------------------------
>     *From:* [email-masked] [mailto:[email-masked]]
>     *On Behalf Of *Bert Wijnen (IETF)
>     *Sent:* Wednesday, June 03, 2009 7:29 AM
>     *To:* [email-masked]
>     *Cc:* Ron Bonica
>     *Subject:* [Netconf] New draft-ietf-netconf-partial-lock-08.txt goes
>     to AD andIETF Last Call
> 
>     WG particpants,
>      
>     Balazs has now done so and
>     revision draft-ietf-netconf-partial-lock-08.txt
>     *From:* Bert Wijnen (IETF)
>     *To:* Balazs Lengyel ; netconf mailing list
>     *Sent:* Tuesday, May 19, 2009 8:51 PM
>     *Subject:* Re: [Netconf] Partial-lock issues
> 
>         *From:* Balazs Lengyel
>         *To:* netconf mailing list
>         *Sent:* Tuesday, May 19, 2009 12:01 PM
>         *Subject:* [Netconf] Partial-lock issues
> 
>         Hello,
> ------------------------------------------------------------------------
> 
> _______________________________________________
> Netconf mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/netconf

---

From: "Bert Wijnen \(IETF\)" <[email-masked]>
Date: Wed, 3 Jun 2009 19:13:20 +0200

I really believe that we have worked hard on this document and that it deserves publication.
With the Warning that Balazs added, people can understand what the issues are and
act accordingly. 
If we want t o add partial-commit and/or multiple candidate-configs, then that is fine.
I am looking forward to Internet-Drafts that make a first pass at describing such functions
after which we can discuss if we need a charter update for that (assuming we have enough
WG members to work on such a document).

Bert
partial-lock document shepherd and WG co-chair
  ----- Original Message ----- 
  From: David B Harrington 
  To: 'Bert Wijnen (IETF)' ; [email-masked] 
  Cc: 'Ron Bonica' 
  Sent: Wednesday, June 03, 2009 6:54 PM
  Subject: RE: [Netconf] New draft-ietf-netconf-partial-lock-08.txt goes to AD andIETF Last Call

  Hi,

  I think there is a real problem that exists, and Balaz's reasoning effectively says "let's just ignore the conflict with access control; then we can make believe the problem does not exist." This does not make the problem go away.

  The problem is that the scope of the lock and the scope of the commit are different.
  I believe that for partial lock on candidiate to work, you need a corresponding partial commit.
  This could be handled using an explicit command for partial commit.
  It could be handled by allowing multiple candidiates, each containing a partial config that is specified as a parameter to the partial lock (or to a partial-copy-config), with what is effectively a global lock of the partial config and a global commit of that partial config.
  The lock and the commit need to match scope. If they do not match, you will have problems.

  If the WG chooses to not have a partial commit that corresponds to the partial lock, or multiple candidiate sandboxes that can allow an operator to independently lock, edit, and commit a partial config, then I think the partial lock for candidate should be removed. 

  That is not my preferred solution, because I believe operators will need to lock and modify and commit partial configs, but removing the partial lock from candidiate is probably the easiest for now.

  dbh

----------------------------------------------------------------------------
    From: [email-masked] [mailto:[email-masked]] On Behalf Of Bert Wijnen (IETF)
    Sent: Wednesday, June 03, 2009 7:29 AM
    To: [email-masked]
    Cc: Ron Bonica
    Subject: [Netconf] New draft-ietf-netconf-partial-lock-08.txt goes to AD andIETF Last Call

    WG particpants,

    Thanks you all.
    Bert and Mehmet

    ----- Original Message ----- 
    From: Bert Wijnen (IETF) 
    To: Balazs Lengyel ; netconf mailing list 
    Sent: Tuesday, May 19, 2009 8:51 PM
    Subject: Re: [Netconf] Partial-lock issues

    Bert
      ----- Original Message ----- 
      From: Balazs Lengyel 
      To: netconf mailing list 
      Sent: Tuesday, May 19, 2009 12:01 PM
      Subject: [Netconf] Partial-lock issues

      Hello,

      Balazs

------------------------------------------------------------------------------

  No virus found in this incoming message.
  Checked by AVG - www.avg.com 
  Version: 8.5.339 / Virus Database: 270.12.52/2152 - Release Date: 06/03/09 05:53:00

---

From: Andy Bierman <[email-masked]>
Date: Wed, 03 Jun 2009 10:29:41 -0700

> I really believe that we have worked hard on this document and that it 

I agree.
If NV-memory is handled by the agent, then
partial locks on <running> work great.
Well-understood boundaries (e.g., per-ifEntry)
can easily be managed among N writers.

Andy

>     *From:* David B Harrington <mailto:[email-masked]>
>     *To:* 'Bert Wijnen (IETF)' <mailto:[email-masked]> ;
>     [email-masked] <mailto:[email-masked]>
>     *Cc:* 'Ron Bonica' <mailto:[email-masked]>
>     *Sent:* Wednesday, June 03, 2009 6:54 PM
>     *Subject:* RE: [Netconf] New draft-ietf-netconf-partial-lock-08.txt
>     goes to AD andIETF Last Call
> 
>     Hi,
>      
>     effectively says "let's just ignore the conflict with access
>     control; then we can make believe the problem does not exist." This
>     does not make the problem go away.
>      
>     It could be handled by allowing multiple candidiates, each
>     containing a partial config that is specified as a parameter to the
>     partial lock (or to a partial-copy-config), with what is effectively
>     a global lock of the partial config and a global commit of that
>     partial config.
>     If the WG chooses to not have a partial commit that corresponds to
>     the partial lock, or multiple candidiate sandboxes that can allow an
>         *From:* [email-masked]
>         [mailto:[email-masked]] *On Behalf Of *Bert Wijnen (IETF)
>         *Subject:* [Netconf] New draft-ietf-netconf-partial-lock-08.txt
>         goes to AD andIETF Last Call
> 
>         WG particpants,
>          
>             consensus and follow it. PLEASE INDICATE WHETHER
>             PARTIAL-LOCKING ON CANDIDATE SHOULD STAY OR
>             SHOULD BE REMOVED !!!!
> 
>             Balazs
> 
>

---

From: Balazs Lengyel <[email-masked]>
Date: Thu, 04 Jun 2009 09:37:46 +0200

Hello,
I have to repeat, that the conflict is NOT the result of partial-locking, it is between access 
control and commit. So why should partial-lock be responsible for solving access control issues?

Unless the chairs and the AD agree with my reasoning, I propose to immediately remove 
partial-lock for candidate and startup. While I stand by my earlier arguments, and disagree 
with David, I want to settle this issue fast, and rather remove startup and candidate, then 
keep on arguing.

However remember, that restricting partial-lock to running, will not make the access 
control/commit conflict go away.

Balazs

> effectively says "let's just ignore the conflict with access control; 

BALAZS: No my reasoning says, that you should not solve access control issues in a locking 
draft. Or do you claim, that partial-locking makes the situation worse. If yes please tell me why!

As described in my earlier mail, you can have the same deadlock without partial-lock, so will 
we remove the commit operation, because it can lead to dead-locks :-)

> The problem is that the scope of the lock and the scope of the commit 

BALAZS: No the problem is with access control. If the user has all needed access rights, there 
will be no dead-locks, or do you think thats not true?

-- 
Balazs Lengyel                       Ericsson Hungary Ltd.
System Manager
ECN: 831 7320                        Fax: +36 1 4377792
Tel: +36-1-437-7320                  email: [email-masked]

---

From: Balazs Lengyel <[email-masked]>
Date: Thu, 04 Jun 2009 09:40:32 +0200

> document first -- one that does not ignore access control.

I agree with Andy, that introducing multiple candidate configs or a partial-commit operation is 
out of scope for now, and requires careful consideration and a recharter.
Balazs

---

From: "Romascanu, Dan (Dan)" <[email-masked]>
Date: Thu, 4 Jun 2009 10:00:53 +0200

I would like to make progress on this issue and forward the document to
IETF Last Call. My reading from the discussions on the list supports the
chairs opinion about consensus, but also shows that at least one strong
and articulated different opinion exists. This is not reflected in the
current PROTO write-up, so I would like to ask Bert to mention the
minority concerns in section (1e) of the write-up (How solid is the WG
consensus behind this document?). 

DBH and other - please feel free to re-iterate your concerns as IETF LC
comments. 

Dan

> -----Original Message-----
> From: [email-masked] 
> [mailto:[email-masked]] On Behalf Of Balazs Lengyel
> Sent: Thursday, June 04, 2009 10:41 AM
> To: Andy Bierman
> Cc: [email-masked]; 'Ron Bonica'
> Subject: Re: [Netconf] New 
> draft-ietf-netconf-partial-lock-08.txt goes to AD andIETF Last Call
> 
> Andy Bierman wrote:
> > As I have stated before, I strongly object to changing the NETCONF 
> > database architecture without a complete re-charter and 
> development of 
> > a NETCONF Architecture document first -- one that does not ignore 
> > access control.
> > 
> 
> I agree with Andy, that introducing multiple candidate 
> configs or a partial-commit operation is out of scope for 
> now, and requires careful consideration and a recharter.
> Balazs

---

From: "Bert Wijnen \(IETF\)" <[email-masked]>
Date: Thu, 4 Jun 2009 12:31:23 +0200

I will update the proto-writeup later today

Bert
  ----- Original Message ----- 
  From: Romascanu, Dan (Dan) 
  To: Balazs Lengyel ; Andy Bierman 
  Cc: Ron Bonica ; [email-masked] 
  Sent: Thursday, June 04, 2009 10:00 AM
  Subject: Re: [Netconf] New draft-ietf-netconf-partial-lock-08.txt goes toAD andIETF Last Call

  Dan

---

From: "Randy Presuhn" <[email-masked]>
Date: Thu, 4 Jun 2009 12:22:09 -0700

Hi -

> From: "Balazs Lengyel" <[email-masked]>
> To: "David B Harrington" <[email-masked]>
> Cc: "'Ron Bonica'" <[email-masked]>; <[email-masked]>
> Sent: Thursday, June 04, 2009 12:37 AM
> Subject: Re: [Netconf] New draft-ietf-netconf-partial-lock-08.txt goes to AD andIETF Last Call
>
> Hello,

If one believes that access decisions are made during "commit" then
all these horrible things *will* happen, and it's broken, just as Balazs argues.

If access decisions are *not* made during "commit"  then the problem goes away.

The problem, if I recall an off-line discussion correctly, is the language in an
example in the netconf spec that indicates that access decisions are made
during the commit.  Other than that, I see no reason for access decisions to
be made during commit time in a single-candidate view of how things should
work.

Randy

---

From: WashamFan <[email-masked]>
Date: Fri, 05 Jun 2009 11:00:59 +0800

Hi,

> solving access control issues?

I admit partial-locking does not make the things worse, and without
partial-locking, the issues are still there. But on the other hand,
users might benefit little from partial-locking with these issues. And
partial-locking gives an impression it encourages simultaneous-edits
on candidate, which is the key reason of access control issues.

If we benefit little from partial-locking on candidate, then removing
the use case seems OK.

>  control/commit conflict go away.

I have no idea why the problem happens on running, since commit
does not apply to running.

>  Balazs

washam

---

From: WashamFan <[email-masked]>
Date: Fri, 05 Jun 2009 11:06:16 +0800

Hi,

>  If access decisions are *not* made during "commit"  then the problem 

I am afraid access decision during commit or not is not the key
reason to the problem. even access decision during config-edit
can cause the problem. The point is, we only edit a portion of
candidate, when we try to commit the whole candidate, we
have no idea if other parts have been modified by other people.

washam

---

From: Balazs Lengyel <[email-masked]>
Date: Fri, 05 Jun 2009 13:51:49 +0200

Hello Randy,
Some other assumptions that might impact this issue:
- Do we have separate/differing access rights for running and candidate? (I hope not.)
- Your question stated somewhat differently: Do we have access control at edit-config/commit or 
both?
- for commit is the access control allowing the commit operation or the modification of the 
relevant data

We will run into some other interesting issues with access control
- how do you use validate (even for your own little area), if you don't have access rights to 
the full config? If validation can take into account areas that are non-readable for the user, 
that may lead to information leaks.
- can you lock the datastore if you are not allowed to read/write parts of it?

But all these are access control issues NOT partial-locking related.

Balazs

---

From: WashamFan <[email-masked]>
Date: Sat, 06 Jun 2009 10:58:55 +0800

Hi,

----- Original Message -----
From: Balazs Lengyel <[email-masked]>
Date: Friday, June 5, 2009 7:52 pm
Subject: [Netconf] Access control [was: ew draft-ietf-netconf-partial-lock-08.txt goes to	AD
To: Randy Presuhn <[email-masked]>
Cc: [email-masked]

> that may lead to information leaks.

I review sec8.6.4.1, rfc4741, which tells me <source> parameter
of <validate> operation might indicate configuration subtree
rather than the full candidate. Although I am not sure if it is the
intent.

washam

---

From: Andy Bierman <[email-masked]>
Date: Fri, 05 Jun 2009 20:24:59 -0700

> Hi,
....
> rather than the full candidate. Although I am not sure if it is the

Actually, the text in this section clearly says
that an empty element for the name of a database is used,
such as <candidate/> or <running/>.  Except the
complexType 'rpcOperationSourceType' in the XSD
does not match this text, because it includes
the 'inline config' option in the choice.

The NETCONF WG needs to decide (for 4741-bis) whether the
the text or the XSD is correct.

The intent of the WG at the time was not really clear,
except that the entire config was being validated
by this operation, not a piece of a config.

If an inline config element was used, then it
would need to be validated as if it were a complete
config database, so must/mandatory top-level nodes not
included in the inline config would still cause
a validation failure.

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Sat, 06 Jun 2009 23:28:54 +0200 (CEST)

Hi,

> rather than the full candidate. Although I am not sure if it is the

This issue has been up a couple of times, and I started a thread
"4741bis - validate" on this issue 04/30.  So far it has received very
few comments.

/martin

---

From: Balazs Lengyel <[email-masked]>
Date: Mon, 08 Jun 2009 08:55:18 +0200

Hello Martin,
My opinion and IMHO the consensus so far is:
- Validate only for a complete database
- test-only option for edit-config
Balazs